24 June 2009 4D Imaging and Tracking Alan Greenaway

24 June 2009 http://waf.eps.hw.ac.uk/ 1

4D Imaging and Tracking

Alan Greenaway


Imaging


Basic principles

• Derived from wavefront sensing in astronomical adaptive optics

• Conventional imaging system gives in-focus image of a single object plane

IM AGE


Basic principles

• Conventional imaging system gives in-focus image of a single object plane

• Combined with conventional grating gives multiple images of single object plane

Diffrac tion Order

-1 0 + 1

IM AG E


Diffractive optics

• Distorted grating gives different phase shift in each diffraction order

• Principle of detour phase holography

• Quadratic distortion wavefront curvature

• Acts like lens with different focal length in each diffraction order

Off-axis Fresnel lens


3-D Snapshot Imaging

-1

0

+1

-1 0 +1

Simple & cheap to manufactureGood control of divergencePhase grating etch gives energy- balance controlHigh optical efficiency from binary grating

Blanchard & Greenaway

App.Opt. 38(1999)6692


3-D Snapshot Imaging

-1

0

+1

-1 0 +1

Simple & cheap to manufactureGood control of divergencePhase grating etch gives energy- balance controlHigh optical efficiency from binary grating

In-focus images of various z-planes are at different magnification


Telecentricity

-1

0

+1

Combination optical system

2

1 2 and and 0cm g

fs f f f p p

f

1 2

m

and and

m g

cm g g m g

f f s f ssff p p

f f s f f s f f s

Djidel, Gansel, Campbell & Greenaway, Opt Exp 14(2006)8269-8277


Telecentricity

-1

0

+1

Telecentric performances f

0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.1

-4.2

-4.1

-4

-3.9

-3.8

-3.7

-3.6

-3.5

Lens to Off-Axis Fresnel Lens Separation (m)

Mag

nif

icat

ion

m+ theoretical

m- theoretical

measured data


20 0 40 0 60 0 80 0 10 00

0 .2

0 .4

0 .6

0 .8

1 .0

Practicality & Efficiency?

Practicality• Used on inverted

and upright systems

Efficiency• Optical efficiency

~84% overall

• (~28% each order)nm etch depth


First Biology Images

GFP in plant-cell mitochondria bright filed (upper row) and epi-fluorescence (bottom row)

1µm separation between in-focus

planes (with Logan, St Andrews)

GFP in drosophila ovary

Epi-fluorescence images with 7.3µm between in-focus planes

Principal application microtubule dynamics through EB1 tracks

(with Davis, Oxford)


3D movies

• 9 z-plane phase-contrast images of Hela cells

• Z-plane separation 810nm

• 3 z-plane DIC images from movie of Hela mitosis

With Allan, Manchester


Motility Measurement• Sperm motility from

phase-contract and/or dark-field imaging

• Current approach uses apparent length as a proxy for out-of-plane deformation of tail

• Multi z-plane images provide extra information

(with Kirkman-Brown, Birmingham)


Imaging- summary• 3D ‘snapshot’ images

on commercial microscopes

• All image modes but scanning demonstrated

• 3D tracking (4D data)

Z-plane separations 0 to many microns

Simultaneous z-stacks Up to 9 simultaneous

planes demonstrated Low-cost add on

Work on sperm motility starts in September


Broadband Application

• Careful design achieve ‘white light’ imaging

• Unfiltered halogen light

Blanchard & Greenaway, Opt. Commun. 183(2000)29-36


Tracking


Ranging in Depth with Sharpness

• Beam divergence from source depends on optical aperture

• Defocused image on non-source planes reduces intensity…

• …thus to a reduction in sharpness (integral of intensity squared)

• Suitable for real-time analysis and CMOS detector technologies 10 12 14 16 18

20

40

60

80

10 0


Experimental Setup

Camera

Iris

Mirror

He-NeLaser

Illumination

Nanoholes

x, y, zNano-

positioners

Objective

f = 200 mm f = 215 mm

Telecentric: Magnification equal in all diffraction orders

215 mm

Experimental arrangement


• 210 nm diameter holes in Al foil

• Single point source

• Simulates fluorescent particle

• Mask / hole contrast >104

• Brightness limited only by illumination source

Nanohole test objects25 m

1.5 m


Resolution on 3 planes-1 order in focus

+1 order in focus

0th order in focus

+1 order in focus

0th order in focus

-1 order in focus

+1 order in focus

0th order in focus

Image Resolution: No grating = 233nmWith grating = 226nm and 231nm (for 0th and ±1orders)

-1 order in focus

+1 order in focus

0th order in focus


Sharpness

Problem:

A single mage sharpness measurement gives ambiguous depth position

-6 -5 -4 -3 -2 -1 0 1 2 3 4

101

102

Nanohole Displacement (m)

Imag

e S

harp

ness

Image Sharpness vs. Nanohole Displacement


Unique depth indication…

-6 -5 -4 -3 -2 -1 0 1 2 3 4

101

102

Nanohole Displacement (m)

Imag

e S

harp

ness

Image Sharpness vs. Nanohole Displacement

-1 Order

0th Order+1 Order

Solution:

QD grating method gives 3 simultaneous image sharpness (one from each order) for each particle.

2 or more images resolve ambiguity


ML depth estimator


Principle

• For any given z-position of source– PDF for sharpness

dependence on photon flux and optics is ‘known’

– Particle depth is ‘most likely’ value that is consistent with measured data – choose z to maximise

2P s z

1P s z

2s 1s

2P s z

1 2P s z P s z


o o o o

o o o o

o o o o2 .0 2 .5 3 .0 3 .5 4 .0 4 .5

0 .02

0 .04

0 .06

0 .08

0 .10

Sharpness SNR

Solid – photon bias

Dotted – actual sharpness

Circles – photon subtractedDainty & GreenawayJOSA 69(1979)786-790

Log n


What is possible?

• MVB model

• SNR per image

• Many approximations – check…

2 0 2 4

0 .004

0 .005

0 .006

0 .007

0 .008

0 .009

0 .01

0 .011

2

221

1

8 jm

jj

MVB zn

z zm

a z z

nSource z-position


Tracking - summary

• Simple algorithm tracking points with high resolution

• More analytic work needed

• Cluttered backgrounds yet to be tried

+8.1nm demonstrated in first experiment

nm depth resolution with ~thousand detected photons

Ad-hoc calibration Work on models

continues


Thanks…STFC for PIPPS/Bio-Mini-PIPSS funding… EPSRC bio-photonics platform funding…

P.A. Dalgarno1, H.I.C. Dalgarno1 , R.J. Warburton1, A. Putod1, S. Aitken1, A Weis1, C Diez1, Alan Baird1

D.P. Towers2, C.E. Towers2, N.C Angarita-Jaimes2, S Chen2

D.J. Smith3, J.C. Kirkman-Brown3, J. Gaham4,V J Allan5, R Southern5, P Marsh5,I Davis6, R Parton6, K Lillie6

1 Dept Physics, SUPA/IIS, Heriot-Watt University, UK2 School of Mechanical Engineering, University of Leeds, Leeds, UK3 Centre for Human Reproductive Science, Birmingham Women’s NHS Foundation Trust,

Edgbaston Birmingham, UK4 Cairn Research Ltd, UK5 Faculty of Life Sciences, University of Manchester6 Dept of Biochemistry, University of Oxford


Questions??

Documents

24 June 2009 4D Imaging and Tracking Alan Greenaway