RANDALL

molecular biophysics

division of cell and

Principles of Principles of

Optical MicroscopyOptical Microscopy

Varenna 2010

Rainer HeintzmannRainer Heintzmann,,

Institute of Photonic Technology (IPHT),Institute of Photonic Technology (IPHT),

Friedrich Schiller University of JenaFriedrich Schiller University of Jena

Randall Division, KingRandall Division, King‘‘s College Londons College London

heintzmann@gheintzmann@googlemail.comooglemail.com

OverviewOverview

�� Physics of LightPhysics of Light

��Microscope Image FormationMicroscope Image Formation

�� Fourier Optics: The Abbe limit Fourier Optics: The Abbe limit

�� SamplingSampling

OverviewOverview

�� Physics of LightPhysics of Light

��Microscope Image FormationMicroscope Image Formation

�� Fourier Optics: The Abbe limit Fourier Optics: The Abbe limit

�� SamplingSampling

Physics of LightPhysics of Light

Light as a ray Light as particle (Photons)

focal distance

plane of focus

Physics of LightPhysics of Light

Light as a Wavefocal distance

plane of focus

Moving Phase Front

Optical Aberrations: Spherical AberrationOptical Aberrations: Spherical Aberration

http://en.wikipedia.org/wiki/File:Spherical_aberration_2.svg

Perfect Lens

Real Lens

htt

p:/

/en.w

ikip

edia

.org

/wik

i/S

pher

ical

_ab

erra

tion

Optical Aberrations: Spherical AberrationOptical Aberrations: Spherical Aberration

http://www.olympusmicro.com/primer/java/aberrations/pointspreadaberration/index.html htt

p:/

/en.w

ikip

edia

.org

/wik

i/F

ile:

Spher

ical

-aber

rati

on-s

lice

.jpg

Interference

incoming wave

scattered wave

total outgoing wave

Interference

incoming wave

scattered wave

total outgoing wave

Interference

incoming wave

scattered wave

total outgoing wave

Phase shift leads to destructive interference!

The modern microscope: Infinity opticsThe modern microscope: Infinity optics

fObj fObj

sample p

lane

back

focal p

lane

imag

e plan

e

fTL fTL

Objective Lens

Tube Lens

infinity path : Filters do not hurt

M = fTL / fObj

OverviewOverview

�� Physics of LightPhysics of Light

��Microscope Image FormationMicroscope Image Formation

�� Fourier Optics: The Abbe limit Fourier Optics: The Abbe limit

�� SamplingSampling

Rainer Heintzmann13

The Complex PlaneThe Complex Plane

real

imaginary

1-1

i = √-1

a

b

ibac +=

( ) ( )[ ]ϕϕϕ sincos iAeAci +==

ϕA

Rainer Heintzmann14

The Complex WaveThe Complex Wave

real

imaginary

x

Wavenumber: k [waves / m]

)( ϕ+kxi

k ea

Cak ∈

[ ] ( )ϕϕ +=+kxaea k

kxi

k cosRe )(

x

Rainer Heintzmann15

Frequency space:Frequency space:

k [1/m]k [1/m]

x [m]x [m]

Real space:Real space:In

ten

sity

Inte

nsi

tyA

mp

litu

de

Am

pli

tud

e

Excurse:Excurse: Spatial FrequenciesSpatial Frequencies

Rainer Heintzmann16

from:: http://members.nbci.com/imehlmir/

Even better approximation:Even better approximation:

Fourier AnalysisFourier Analysisfrom:: http://www-groups.dcs.st-and.ac.uk/

~history/PictDisplay/Fourier.html

Rainer Heintzmann17

The Running WaveThe Running Wave

xkixkieAeAxA

vvvvv

0

,

0)( ==

xkvv

,

xv

k

k

v

v

2

0

*

2

)()(

)()(

))(Re(~)(

A

xAxA

xAxI

xAxE

=

=

=vv

vv

vv

Rainer Heintzmann18

Constructing images from wavesConstructing images from waves

Sum of Waves

Corresponding

Sine-Wave

Object:

kx

ky

kx

ky

Accounted

Frequencies

Spatial

Frequency

Rainer Heintzmann19

Constructing Images from WavesConstructing Images from Waves

Accounted Frequencies Sum of Waves

Spatial Frequency Corresponding WaveObject:

kx

ky

kx

ky

Rainer Heintzmann20

Fourier FilteringFourier Filtering

kkxx

kkyy

Fourier Fourier

domaindomain

Real spaceReal spaceFourier Fourier

domaindomain

DFTDFT

suppresssuppress

high spatialhigh spatial

frequenciesfrequencies

kkxx

kkzzkkzz

0

1

kkxx

Rainer Heintzmann21

FourierFourier--transformationtransformation & & OpticsOptics

•Plane Waves are simple points in

reciprocal space

•A lens performs a Fourier-transform

between its FociFourier-transformation of Amplitude

Rainer Heintzmann22

FourierFourier--transformationtransformation & & OpticsOptics

Fourier-

planeObject Image

f f f f

Laser

Rainer Heintzmann23

Larger Region in Larger Region in Frequency SpaceFrequency Space

→→ Higher ResolutionHigher Resolution

Resolution criteriaResolution criteria

Rainer Heintzmann, 201025

Resolution Criteria (Rayleigh)Resolution Criteria (Rayleigh)

“Airy” Disc X Position

Inte

nsi

ty

“First

Intensity

Zero”

Problems:

No Zero at high NA (vectorial theory)

Zero position can be shifted to anywhere at the

expense of high side-lobes (see Toraldo Filter)

Defined for Point Objects

LIMIT IS NOT ABSOLUTE

Rainer Heintzmann, 201026

Resolution Criteria (Sparrow)Resolution Criteria (Sparrow)

X Position

Inte

nsi

ty

“No Dip”

Problems:

Defined for Point Objects

LIMIT IS NOT ABSOLUTE

(Images cannot be decomposed into point pairs)

Resolution Criteria (Abbe)Resolution Criteria (Abbe)

Back to Theory:

Light is Waves !

Resolution Criteria (Abbe)Resolution Criteria (Abbe)

Back to Theory:

Light is Waves !Resolution depends on

diffraction angle

(wavelength,

refractive Index)

and

lens acceptance angleα

Abbe’s

Resolution Lim

it

Fluorescence imagingFluorescence imaging

FluorescenceFluorescence –– JablonskiJablonski DiagramDiagram

~σIex

ps

ps

∼τ –1

Singlet Triplet

Electronic Excited State

Electronic Ground State

Atomic Nuclei

Electron Cloud

Local Temperature:

~1000 deg

Stokes Shift

Fluorescence:Fluorescence:

Incoherent superposition of intensities!Incoherent superposition of intensities!

Each wave-pair

forms a sine waves

in the amplitude

image.

Intensity is Airy

pattern

Abbe and FluorescenceAbbe and Fluorescence

Back to Theory: Light is Waves,

BUT Fluorescence looses the phase

information,

Independent emitters,

no interference with other emitters

Abbe’s Limit

…… Suppose sample is a sineSuppose sample is a sine--wavewave

Object (Sine wave):

Microscope Image (ALSO a sine wave):

Inte

nsi

ty

Spatial Coordinate

Inte

nsi

ty

Spatial Coordinate

contr

ast

k

1limit

||kkx,yx,y| |

[1/m][1/m]

ImageImage Sine Sine

Wave StrengthWave Strength

Abbe limitAbbe limit

00

11

Abbe Limit in Fluorescence EmissionAbbe Limit in Fluorescence Emission

Object

Sine Wave

Frequency

The Optical Intensity Transfer Function

Abbe's limitAbbe's limitdoes not does not drescribedrescribewidth !!!width !!!

NAnD

2sin2min

λα

λ==

Example 1:Example 1:

NAnD

2sin2min

λα

λ==

back aperture

CTFOTF PSF

NAFWHM

λ42.0=

min

2

D

π

Example 2:Example 2:

NAnD

2sin2min

λα

λ==

back aperture

CTFOTF PSF

NAFWHM

4

λ=

FWHMDmin

Example 3:Example 3:

NAnD

2sin2min

λα

λ==

back aperture

CTFOTF PSF

phase rings

FWHM

NAFWHM

10

λ=

Toraldo Filter (Toraldo di Francia): Nuovo Cimento, Suppl. 9 (1952) 426-438

How to circumventHow to circumventAbbe's Abbe's

frequency limit ?frequency limit ?

Wait a minute Wait a minute ……..

We can excite AND detect fluorescence independently

Abbe’s Limit

||kkx,yx,y| |

[1/m][1/m]

Exci

tati

on

Exci

tati

on

Sin

e S

ine

Wav

e S

tren

gth

Wav

e S

tren

gth ExcitationExcitation

Abbe limitAbbe limit

00

11

Abbe Limit in Fluorescence Ex + Abbe Limit in Fluorescence Ex + EmEm

Excitation

Sine Wave

Frequency

Optical Excitation Transfer Function

||kkx,yx,y| |

[1/m][1/m]

Imag

eIm

age

Sin

e W

ave

Sin

e W

ave

Str

ength

Str

ength

EmissionEmission

Abbe limitAbbe limit

00

11

Emission

Sine Wave

Frequency

FourierFourier--spacespace& &

OpticsOptics

Intensity in Focus (PSF)Intensity in Focus (PSF)

Reciprocal Space(ATF)

kx

kz

ky

Real Space(PSF)

x

z

y

Lens

Focus

Oil

Cover Glass

EpifluEpifluoorescentrescent PSFPSF

?

I(x) = |A(x)|2 = A(x) · A(x)*

I(k) = A(k) ⊗ A(-k) OTF

ATF

~ ~ ~Fourier Transform

ConvolutionConvolution: : DrawingDrawing withwith a a BrushBrush

kx,y

kzRegion of Support

Optical Transfer Function (OTF)Optical Transfer Function (OTF)

!

kx,y

kz

Widefield OTF supportWidefield OTF support

Missing Cone

Confocal Confocal Fluorescence Fluorescence Microscopy:Microscopy:

Excite AND DetectExcite AND Detect

Rainer Heintzmann49

The Widefield MicroscopeThe Widefield Microscope

xxyy

zz

PinholePinhole

PinholePinholePMTPMT

The Confocal MicroscopeThe Confocal Microscope

ZZ

I(Z)I(Z)

Standard Lightsource

Camera

Standard Lightsource

Camera

LightsourceLightsource

with Pinholewith Pinhole

Rainer Heintzmann50

Confocal OTFConfocal OTF

kx,y

kz

Excite AND Detect: P(r) = PEmission(r) PDetection(r)

PSF(r) = PSFEmission(r) PSFDetection(r)

OTF(k) = OTFEmission(r) ⊗ OTFDetection(r)

kx,y

kz

α

Increasing the aperture angle (α) enhances resolution !!

We have circumvented We have circumvented Abbe:Abbe:

NAnD

4sin4confocalmin,

λα

λ=≈

Confocal Confocal OTFsOTFs: : ��

WF

1 AU

0.3 AU

in-plane, in-focus OTF

1.4 NA Objective

WF Limit

New Confocal Limit

Almost no transfer

OverviewOverview

�� Physics of LightPhysics of Light

��Microscope Image FormationMicroscope Image Formation

�� Fourier Optics: The Abbe limit Fourier Optics: The Abbe limit

�� SamplingSampling

54

Correct SamplingCorrect Sampling

What is SAMPLING?What is SAMPLING?

Intensity [a.u.]

2 3 4 5 6 X [µm]1

Aliasing Aliasing …… suppose it is a sinesuppose it is a sine--wavewave

Intensity [a.u.]

2 3 4 5 6

There are many sine-waves,

SAMPLED with the same measurements.

Which is the correct one?

Aliasing Aliasing …… suppose it is a sinesuppose it is a sine--wavewave

… maybe we can know!Object:

Microscope Image:

Inte

nsi

ty

Spatial Coordinate

Inte

nsi

ty

Spatial Coordinate

Aliasing Aliasing in Fourierin Fourier--spacespace

Fourier-transform of Image

Inte

nsi

ty

Aliased Frequencies

½ Sampling Frequency

½ Nyquist Frequencies

Pixel sensitivityPixel sensitivity

Intensity [a.u.]

2 3 4 5 6 X [µm]1

Convolution of pixel form factor

with sampling positions

⇒ Multiplication in

Fourier-space

⇒ Reduced sensitivity at high

spatial frequency

OOptical ptical TTransfer ransfer FFunctionunction

||kkx,yx,y| |

[1/m][1/m]

contrastcontrast

CutCut--off limitoff limit

00

11rectange form-factor

specimen

sampled

Consequences of high samplingConsequences of high sampling

Confocal: high ZoomConfocal: high Zoom →→ moremore bleachingbleaching??

No! No!

ifif laserlaser isis dimmeddimmed oror scanscan--speedspeed adjustedadjusted

→→ bad bad signalsignal to to noisenoise ratioratio??YesYes, , butbut photonphoton positionspositions areare onlyonly

measuredmeasured moremore accuratelyaccurately

→→binningbinning still still possiblepossible →→high SNR.high SNR.

ReadoutReadout noisenoise isis a a problemproblem

at high at high spatialspatial samplingsampling (CCD)(CCD)

62

Optimal Sampling?Optimal Sampling?

Regular samplingRegular sampling

Reciprocal δ-Sampling GridReal-space sampling:

Multiplied in real space

with band-limited information

Widefield SamplingWidefield Sampling

⇒In-Plane sampling distance

⇒ Axial sampling distance

obj

em

xyNA

d4

max,

λ=

( ))cos(1

)sin(

2max,

obj

obj

obj

em

zNA

dα

αλ−

=

Confocal SamplingConfocal Sampling

⇒In-Plane sampling distance (very small pinhole)

else use widefield equation

⇒ Axial sampling distance

( ))cos(1

)sin(

2max,

obj

obj

obj

eff

zNA

dα

αλ

−=

emex

eff

λλ

λ11

1

+=

obj

eff

xyNA

d4

max,

λ=

Confocal Confocal OTFsOTFs

WF

1 AU

0.3 AU

in-plane, in-focus OTF

1.4 NA Objective

WF Limit

Hexagonal samplingHexagonal sampling

Advantage: ~17%Advantage: ~17%

+ + lessless ‚‚almostalmost emptyempty‘‘ informationinformation collectedcollected

+ + lessless readoutreadout--noisenoise

approximationapproximation in confocal; 3D: ABA, ABC in confocal; 3D: ABA, ABC stackingstacking

Reciprocal δ-Sampling GridReal-space sampling:

Multiplied in real space

with band-limited information

6363×× 1.4 NA Oil Objective1.4 NA Oil Objective (n=1.516),(n=1.516),

excitationexcitation at 488 nm, at 488 nm, emissionemission at 520 nm at 520 nm ⇒⇒ λλeffeff = 251.75 nm, = 251.75 nm, αα = 67.44 = 67.44 degdeg

widefield widefield inin--planeplane: : ddxyxy < < 92.8 nm92.8 nm

⇒⇒ maximal CCD maximal CCD pixelsizepixelsize: 63: 63××92.8 = 5.85 92.8 = 5.85 µµm m

confocal confocal inin--planeplane:: ddxyxy < < 54.9 nm54.9 nm

widefield axial: widefield axial: ddzz < < 278.2 nm278.2 nm

confocal axial: confocal axial: ddzz < < 134.6 nm134.6 nm

Fluorescence Sampling ExampleFluorescence Sampling Example

OTF is not zero but very smallOTF is not zero but very small(e.g. confocal in(e.g. confocal in--plane frequency)plane frequency)

Object possesses no higher frequenciesObject possesses no higher frequencies

You are only interested in certain frequenciesYou are only interested in certain frequencies

(e.g. in counting cells serious under(e.g. in counting cells serious under--sampling is sampling is acceptable)acceptable)

Reasons for Reasons for UndersamplingUndersampling

Detector generates highDetector generates high--frequency noise?frequency noise?

⇒⇒ Measure this noise Measure this noise

(e.g. dark exposure and 2D FFT)(e.g. dark exposure and 2D FFT)

⇒⇒ Avoid aliasing by sampling above this Avoid aliasing by sampling above this

noise frequency.noise frequency.

Traps and PitfallsTraps and Pitfalls

FFT of dark CCD exposure (2 FFT of dark CCD exposure (2 µµs)s)

If you need If you need

high resolution high resolution

or need to detect or need to detect

small samplessmall samples

→→ sample your image correctly along sample your image correctly along

all dimensionsall dimensions

Sampling SummarySampling Summary

Rainer Heintzmann, 201073

Future: We need you!Future: We need you!PhDPhD or or PosdocPosdoc in Jena, Germany !in Jena, Germany !

heintzmann@gmail.comheintzmann@gmail.com

Institute forInstitute forPhotonic Photonic TechnologiesTechnologies

Friedrich SchillerFriedrich SchillerUniversity of JenaUniversity of Jena