PIBM Sept 05 Andy Harvey 1 Spectral Imaging of the Retina Andy R.Harvey, Ied Abboud, Alistair Gorman, Andy I.McNaught * School of Engineering & Physical

1PIBM Sept 05 Andy Harvey

Spectral Imaging of the Retina

Andy R.Harvey, Ied Abboud, Alistair Gorman, Andy I.McNaught*

School of Engineering & Physical Sciences,

Heriot Watt University, Edinburgh, UK

*Eye Unit,Cheltenham General Hospital,

Cheltenham, UK


Outline

• What is spectral imaging?• Spectral retinal imaging

• Why?• Spectral time-sequential spectral imaging

• For flexibility and research

• 2D snapshot• For real-time, high throughput screening

• Conclusions


Conventional Spectral Imaging

RGB Image Spectrally classified image

Dysplasticcell

Superficialsquamouscell

Intermediatecell

Lymphocyte

PMN

Courtesy CRI

Classificationspectra


Imaging the eye

Sclera

Cornea

Iris

Lens

Retina

Macula

Vitreous humour

Light

Anterior chamber (full of aqueous humour)

Optic nerve

Posterior chamber

Choroid

Optic disc


The Role of Spectral Retinal Imaging

• By 2020 there will be 200 million visually-impaired people world wide• Glaucoma, diabetic retinopathy, ARMD• 80% of those cases are preventable or

treatable • Screening and early detection are

crucial • Can spectral imaging offer

enhancements to current screening techniques ?

• Spectral imaging is non-invasive and safe• cf. fluorescein angiogram

• Spectral imaging can enable imaging of • Retinal biochemistry

• Blood oximetry• Diabetic retinopathy, glaucoma

• Lipofuscin etc• Age-related macula degeneration


Spectral Imaging:Traditional approaches

And Fourier-transform equivalents

N(t)

NxNy

N

NxNy(t)

Time-sequential spectral multiplex

Time-sequential spatial multiplex

• Limitations

• Optically inefficient

• 2D time-varying scenes

• 2D snapshot

• required for:• Retinal

imaging• in vitro, in

vivo imaging


Spectral Fundus Camera

• Source filtering by LCTF incorporated into COTS fundus camera• 10 nm spectral width• 20 msec random access

• Images captured using a cooled, low-noise CCD camera


1.00

10.00

100.00

1000.00

10000.00

200 400 600 800 1000

Wavelength (nm)

Abs

orpt

ion

Coe

ffici

ent (

cm-1

)

Absorption Coefficient for HbO2 (cm-1)

Absorption Coefficient for Hb (cm-1)

Isobestic point

Coregistered Spectral Images of a Healthy Retina

800nm

Images translationally and rotationally coregistered


Spectral angle map of healthy and diabetic retina

• Shading indicates similarity of each pixel spectrum with artery and vein spectra• Qualitative oxymetry

Normal Retina

Diabetic Retina


Supervised spectral classifiaction

• Implicit calibration based on spectral signatures within the eye• Classification possible

without absolute calibration

• Clear distinction between veins/arteries, on/off optic disc • Spectra depends on

local environment

• Inversion of data to calculate biochemical concentrations (eg oxygenation) requires a model of light propagation and scattering in the retina to remove environmental effects

• Monte Carlo, Kubelka Monk, Transfer equation


Requirements for a snapshot technique: retinal imaging

• Improved calibration

• Patient patience

• Remove imperfect coregistration

• due to Variations in imaging distortion between images

• Similar issues with other in vivo applications

• Imaging internal epithelial cancers

• Eg gastrointestinal

PC15


Image Replication Imaging Spectrometer: IRIS

Snapshot image• zero temporal misregistration

• ‘100%’ optical efficiency• World’s only snapshot, 2D

spectral imager (almost !)• Conceptually related to Lyot

filter

Large formatdetector

SpectralDemultiplexor


Lyot filter: principle of operation

n=1 � l Cos2@pîDDCos2@pîDDCos2@2pîDDCos2@pîDDCos2@2pîDDCos2@4pîDDCos2@pîDDCos2@2pîDDCos2@4pîDDCos2@8pîDD

PolariserWaveplate


Exploded view of N Wollaston prisms N wave plates

2N spectral images at detector Field

stop

Input polarizer

• Wollaston prism polarisers replicate images• Each Wollaston prism-waveplate pair provides both cos2 and sin2 responses

• All possible products of spectral responses are formed at detector

)(sin

)(cos2

2

)2(sin

)2(cos2

2

)4(sin

)4(cos2

2

IRIS snapshot spectral imager: principle of operation


Spectral transmission

cos2

sin2

cos2(cos2(2)

sin2 (

cos2 (2)

cos2(sin2(2)

sin2(sin2(2)

cos2(cos2(2)cos2(4) cos2(sin2(2)cos2(4)

cos2(cos2(2)sin2(4) cos2(sin2()sin2(4)

sin2(cos2(2)cos2(4) sin2(sin2(2)cos2(4)

sin2(cos2(2)sin2(4) sin2(sin2()sin2(4)

Wollaston/waveplateassembly


Spectral responses

• Bands are overlapping bell shapes• Choose cost function to minimise sidelobes

• Small (~5%) reduction in spectral separation• Cut-off filters used to define spectral range

Theoretical system response

0

20

40

60

80

100

450 500 550 600 650 700 750 800 850

Wavelength (nm)

Res

po

nse

(%

)

•8 channel visible-band system

•520nm820m

•3 Quartz retarders

•32 channel, visible-band system

•520nm 720nm

•5 Quartz retarders


Optical scaling laws

Hamamatsu

ORCA-ER

Inputs:

FoV

Sub image size on CCD

CCD pixel size

Primary lens magnification & F#

Collimating lens back focal distance, focal length & front element diameter

Prism birefringence

Outputs:

Field stop size

Collimating lens rear element diameter

Splitting angles, apertures & depths of prisms

Apertures of retarders, polarisers and filters

Imaging lens focal length & front element diameter

Field stopCollimating

lens

Bandpass

filter

Imaging

lens

Camera

Polariser, retarders & Wollaston prisms

(index matched)Primary lens


Components & Assembly

• 8 channel system• 520nm to 820nm• 3 Quartz retarders• 3 Calcite Wollaston prisms


Spectral Retinal Imaging • Difficult imaging conditions render application of traditional HSI

techniques problematic• IRIS enables real-time and snapshot spectral imaging

Canon CR4-45NMCR4-45NM


Blood oximetry

• Optimal spectral band for retinal oximetry• Vessel thickness ~ optical depth• 570-615 nm• Eight bands approximately equally spaced

0

2

4

6

8

10

12

14

16

18

20

565 575 585 595 605 615 625

Wavelength (nm)

Tra

nsm

issi

on (

%)

40

20

80


Video sequence recorded with bandpass filtered inspection lamp


Retinal image recorded with flash illumination


574581585592595603607613

Coregistered and PCA images

PC1PC2PC1 & PC2


Summary

• Spectral imaging of the retina shows promise for non-invasive detection of retinal disease• Clinical trials on-going

• LCTF-based, time-sequential spectral filtering enables rapid and flexible 2D spectral retinal imaging• Flexible data acquisition• Pulse and other motion artefacts limit accuracy

• Snapshot spectral imaging in 2D (IRIS) promises high-performance real-time multi-spectral imaging• Ideal for in vivo imaging• No temporal misregistration

• Absolute, quantitative data requires a model of light interaction within the retina


• Wollaston prism polarisers replicate images• Each Wollaston prism-waveplate pair provides both cos2 and sin2 responses

• All possible products of spectral responses are formed at detector

Exploded view of N Wollaston prisms N wave plates

2N spectral images at detector Field

stop

Input polarizer

IRIS snapshot spectral imager


Measured & predicted spectral responses


Absolute total transmission

• Bandpass filter & polariser dominate losses

• Improved system: T>80%

• Theoretical throughput is 2n times higher than for spatial/spectral multiplexed techniques!

0

25

50

Re

sp

on

se

(%

)

Absolute response curves in polarised light


Application to microscopy:Imaging of multiple fluorophors

• IRIS fitted to conventional epi-fluorescence microscope

• Germinating spores of Neurospora crassa stained with• GFP – nucleii fluoresce at 510 nm• FM4-64 – membranes fluoresce at >580 nm0

25

50

Re

sp

on

se

(%

)


Principle component decomposition

PC1

PC15

• Artery structure is a pulse artefact

• Very difficult to co-register by image processing means

• Snapshot technique desirable

PC3


Conclusions

• IRIS is a new spectral imaging technique that enables snapshot spectral imaging in 2D• No rejection of light• No data inversion

• Highest-possible signal-to-noise ratios• Simple logistics

• Inherently compact and robust• Simply fitted to conventional imaging systems

• Birefringent materials exist for applications from 0.2m to 12 m

• Applications• In vivo, in vitro imaging

• Retinal imaging• Microscopy

• Multiple fluorophors• Quantum dots

• Surveillance• Remote sensing• Etc.


• Optical depth of Hb & HbO2 dominates variation of penetration with

• Tissues vary between highly turbid and transparent• Blue light images retinal surface• Light at ~600 nm enables spectral oximetry within retinal blood vessels

• optical depth of HbO2 > vessel thickness so vessels translucent

• optical depth of Hb < vessel thickness so vessels are opaque

• Light > 640 nm penetrates to coroid

BlueGreenRed

1.00

10.00

100.00

1000.00

10000.00

200 400 600 800 1000

Wavelength (nm)

Abs

orpt

ion

Coe

ffici

ent (

cm-1

)

Absorption Coefficient for HbO2 (cm-1)

Absorption Coefficient for Hb (cm-1)

Isobestic point

Spectral Characteristics of the Retina


Issues for Spectral Retinal Imaging

• Calibration• Components of interest within a

complex turbid medium• Patient tolerance

• Using current technology, time-sequentialspectral bandpass offers• Optimal SNR• Reduced light intensity at the retina• Agile selection of spectral bands (data efficient)

• Issues• Coregistration• Calibration

± 100 pixels

±2º

• Spectral imaging of static scenes is relatively ‘easy’

• Spectral imaging of the retina encounters

• Imaging through an erratically moving, low-quality f/6 eye-lens system

Solutions: 2D snapshot spectral imaging


The End


1D image x path difference

Fixedmirror

Scanning mirror

Detector array

N

NxNy(t)

N

NxNy(t)

FTFT

N(t)

NxNy

N

NxNy(t)

Direct Imaging Spectrometry (Fourier) Transform Imaging SpectrometryT

emp

oral

ly s

can

ned

Sn

apsh

ot/f

ull

y st

arin

g

N(t)

NxNy

FT

N

NxNy


Why another spectral imaging technique?

• Traditional approaches• Time sequential spectral multiplex

• Monochromatic two-dimensional image in snapshot• Time sequential spatial multiplex

• One-dimensional spectral image in a snapshot• (and Fourier-transform equivalents)

• Problems• Cannot record two-dimensional spectral images of time-varying

scenes• Optically inefficient

• Time-resolved (snapshot) spectral imaging is required for• Dynamic scenes

• In vitro, in vivo imaging and microsocopy• Combustion dynamics, surveillance…

• Irregular motion between scene and imager• In vivo imaging• Ophthalmology• Remote sensing, airborne surveillance, industrial inspection…

Documents

PIBM Sept 05 Andy Harvey 1 Spectral Imaging of the Retina Andy R.Harvey, Ied Abboud, Alistair Gorman, Andy I.McNaught * School of Engineering & Physical