Sharpness Search Algorithms for Automatic Focusing in the …cbatten/pdfs/batten-image... · 2020. 11. 11. · Sharpness Search Algorithms for Automatic Focusing in the Scanning Electron

Sharpness Search Algorithms for Automatic Focusing in the Scanning Electron Microscope

C.F. Batten, D.M. Holburn, B.C. Breton, N.H.M. CaldwellScientific Imaging Group

Cambridge University

ScanningMay 7th, 2001

May 7th, 2001 Scanning 2

Motivation

• There is a growing need for instrument automation– New applications have led to an increase in the

number of novice SEM operators– Remote microscopy requires simple commands

which perform more work

• Focusing is an ideal candidate for automation– Simplifies a common and tedious operation– Helps make remote microscopy practical


Previous Work

• Scanning Electron Microscopy– Software solution using image gradient [Tee79]– Hardware solution using image covariance [Erasmus82] – Software solution using frequency domain [Ong98, Ogasawara99]– Use of a general imaging model to predict best focus [Nicolls95]

• Optical Microscopy– Survey of sharpness measures [Groen85, Firestone91]– Use of a Fibonacci search to find the best focus [Yeo93]


Our Approach

• Traditional autofocusing approaches – Try to integrate additional functionality such as astigmatism

correction or topological mapping– Use a fixed stepsize or iterative search and avoid more

sophisticated search algorithms due to low SNR and hysteresis concerns

• Our approach– Make a dedicated autofocusing search algorithm

Increase the SNR for

each image

Use a more sophisticated

search algorithm

Decrease the number of required

image captures


Outline

• Sharpness Measures– Gradient measure– Frequency domain measure– Autocorrelation measures– Variance measure

• Sharpness Search Algorithms– Fixed stepsize search– Fixed stepsize search with interpolation– Iterative search– Variable stepsize search– Fibonacci search

• Conclusions


Evaluating Sharpness Measures

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40

10

20

30

40

50

60

Focus Step (0.01mm)

Var

ianc

e S

harp

ness

Mea

sure

Forward Sweep 1Forward Sweep 2Forward Sweep 3

Strictly Unimodal Property

Sharpness measure should have one peak at the best focus and strictly decrease away from this maximum


Gradient Measure

• Sum of the difference between every nth pixel in both the X and Y directions

• As image comes into focus, edges become sharper increasing the image gradient

• Sharpness Measure Properties– Relatively easy to calculate (one of the first sharpness measures)– Very susceptible to noise – The parameter n acts a low-pass filter in the spatial domain

• (n = 1) Traditional image gradient• (n = 2) Brenner method• (n > 2) As long as n < feature size, can increase noise robustness


Frequency Domain Measure

• Perform Fourier transform and then sum the frequency components below threshold frequency (Ω)

• As image comes into focus, edges become sharper which increases the magnitude of medium frequency components

• Sharpness Measure Properties– Allows easy integration of astigmatism correction– Fourier transform in software is computationally expensive– The parameter Ω acts as a low-pass filter in frequency domain

• Varying Ω produces similar results as varying n• For this work, Ω chosen to be 50


Auto-correlation Measures

• Auto-correlation function is the image convolved with itself and indicates how well neighboring pixels are correlated

• Tested two measures using the image auto-correlation – ACFdiff Height of the central ACF peak– ACFsum Area under the central ACF peak

• Focused images contain small highly correlated regions that result in a tall sharp central ACF peak

• Sharpness Measure Properties– Can calculate ACF efficiently in the frequency domain– Do not need to calculate entire ACF for ACFdiff measure– Correlated noise due to limited bandwidth distortion made

using the ACF more difficult


Variance Measure

• Sum the square of the difference between each pixel and the mean image intensity

• Focused images have greater intensity variation then blurred defocused images

• Sharpness Measure Properties– Simple and efficient implementation– Very robust to noise– Strong adherence to the strict unimodality property

For these reasons the variance measure was selected as the primary sharpness measure for this work


Comparison of Sharpness Measures

0 10 20 300.4

0.6

0.8

1

(a) fgrad

0 10 20 30−0.5

0

0.5

1

(c) fACFdiff

0 10 20 30

0.2

0.4

0.6

0.8

1

(b) ffreq

0 10 20 300

0.2

0.4

0.6

0.8

1

(e) fvar

Pix Avg: 1 FAvg: 3 Pix Avg: 1 FAvg: 5 Pix Avg: 2 FAvg: 3 Pix Avg: 2 FAvg: 10 Pix Avg: 4 FAvg: 15 Pix Avg: 8 FAvg: 20 Pix Avg: 16 FAvg: 25

0 10 20 300

0.2

0.4

0.6

0.8

1

(d) fACFsum

Sharpness Measures at Various Noise Levels


Sharpness Search Algorithms

• Investigated five sharpness search algorithms– Fixed stepsize search– Fixed stepsize search with interpolation– Iterative search– Variable stepsize search– Fibonacci search

• Notation– l Search interval– α Desired accuracy (How close to optimum is acceptable?)– N Number of required image captures

• Goal is to find a search algorithm which minimizes Nbut still achieves the desired accuracy α


Fixed Stepsize Search

• Single sweep over search interval with stepsize = 2α• Theoretical N given by

• Peak finding reduces N• Developed a novel method

to adjust for hysteresis effects based on relative sharpness when returning to best focus

4 4.5 5 5.5 6 6.5400

500

600

700

800

900

1000

1100

Focal Length (mm)

Var

ianc

e S

harp

ness

Mea

sure

Variance Moving Average

+= 12αlN

Fixed Stepsize Search with Peak Finding


Fixed Stepsize with Interpolation

• Interpolation can help reduce the number of image captures while maintaining the desired accuracy

• Quadratic and linear interpolation do not perform well on typical variance curves

• A New Interpolation Approach– Erasmus and Smith provide a derivation for image variance

as a function of defocus [Erasmus82]– Use non-linear regression to curve fit the derived function

with the collected data– This allows us to significantly reduce the required number of

image captures, but is computationally expensive


Fixed Stepsize with Interpolation

8.5 8.61 8.72 8.83 8.940.2

0.4

0.6

0.8

1

1.2

1.4

Focal Length (mm)

Nor

mal

ized

Var

ianc

e

(a) Gold on Carbon − 22,200x

8.56 8.6 8.65 8.69 8.740.2

0.4

0.6

0.8

1

1.2

Focal Length (mm)

Nor

mal

ized

Var

ianc

e

(b) Gold on Carbon − 75,500x

8 8.66 9.33 10 10.66

0.6

0.8

1

1.2

Focal Length (mm)

Nor

mal

ized

Var

ianc

e

(c) IC Tracks − 3,400x

6 7.55 9.11 10.66 12.220.6

0.7

0.8

0.9

1

1.1

1.2

Focal Length (mm)

Nor

mal

ized

Var

ianc

e

(d) IC Tracks − 3,400x

8.69 8.98 9.27 9.56 9.850.5

0.6

0.7

0.8

0.9

1

1.1

Focal Length (mm)

Nor

mal

ized

Var

ianc

e

(e) IC Tracks − 9,900x

7 8.33 9.66 11 12.330.4

0.6

0.8

1

1.2

1.4

Focal Length (mm)N

orm

aliz

ed V

aria

nce

(f) Sublimated Titanium − 1,500x

Derived Variance Function Fitted to Data from Various Specimens


Iterative Search

• Several sweeps with gradually smaller stepsizes and search intervals

• Theoretical N given by

where η is the number of image captures per iteration

=

1))-η/(2log()/αlog(η lN

6 6.5 7 7.5 8 8.5 9 9.5300

350

400

450

500

550

600

Focal Length (mm)

Var

ianc

e S

harp

ness

Mea

sure

Iteration 1Iteration 2Iteration 3Iteration 4

Online Iterative Focus Sweep (η=8)


Variable Stepsize Search

• Reduce the stepsize as the sharpness increases

• A common technique in other maximum search problems, but not used in SEM autofocusing due to low image SNR

• We adapt the algorithm as follows– Reduce stepsize based on moving average of variance– Set 2α as a lower bound on the stepsize – Use peak finding– Perform final fixed stepsize search if stepsize

is greater than 2α once the peak is found

• Actual number of image captures varies based on initial stepsize and specific variance curve


Variable Stepsize Search

4 4.5 5 5.5 6 6.5400

500

600

700

800

900

1000

1100(a) Sharpness During Variable Stepsize Sweep

Focal Length (mm)

Var

ianc

e S

harp

ness

Mea

sure Variance

Moving Average

4 4.5 5 5.5 6 6.5

0.1

0.15

0.2

0.25

0.3

0.35(b) Stepsize During Variable Stepsize Search

Focal Length (mm)

Ste

psiz

e (m

m)

Example of Variable Stepsize Search


Fibonacci Search

• An iterative search where η = 1• Use previous measurements and one new measurement to

narrow search interval• To avoid adverse hysteresis

effects, must set instrument to small focal length before each image capture (~200ms)

0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34400

500

600

700

800

900

1000

1100

Focal Length (arbitrary units)

Sha

rpne

ss M

easu

re

•

•

•

•

•

••

1

2

3

4

5

6

7

a b

Fibonacci Search Image Captures


Fibonacci Search

7.1 7.15 7.2 7.25 7.3 7.35 7.4200

400

600

800

1000

1200

1400

1600

1800

Focal Length (mm)

Var

ianc

e S

harp

ness

Mea

sure

Fixed Stepsize SearchFibonacci Search

Example of Fibonacci Search


Results: Number of Image Captures

0.010.020.030.040.050.060.070.080.090.10

10

20

30

40

50

60

70

80

90

100

Relative Accuracy (accuracy/search interval)

Num

ber

of R

equi

red

Imag

e C

aptu

res

Fixed Stepsize Search Iterative Search (η = 8) Variable Stepsize Search (Test #1)Variable Stepsize Search (Test #2)Fibonacci Search

Total Search Time vs. Relative Accuracy


Results: Total Search Time

0.0140.0160.0180.020.0220.0240.0260.0280.030.0320

20

40

60

80

100

120

140

160

Relative Accuracy (accuracy/search interval)

Tim

e (s

econ

ds)

Fixed Stepsize Search with Interpolation Iterative Search Variable Stepsize SearchFibonacci Search

Total Search Time vs. Relative Accuracy


Results: Relative Sharpness

• Hysteresis effects prevent us from just comparing the best focus produced by each sharpness search algorithm

• Use relative sharpness as a more accurate metric

(a) Gold on Carbon25,800x

(b) Integrated Circuit970x

(c) Sublimated Titanium1,350x

(d) Etched Silicon410x

Specimens Used for Relative Sharpness Tests


Results: Relative Sharpness

FIX ITP ITR VAR FIB440

460

480

500

520

540(b) Integrated Circuit

Bes

t Foc

us V

aria

nce


1100

1200

1300

1400(c) Sublimated Titanium

Bes

t Foc

us V

aria

nce


400

420

440

460(d) Etched Silicon

Bes

t Foc

us V

aria

nce


1600

1800

2000

2200

2400

2600(a) Gold on Carbon

Bes

t Foc

us V

aria

nce

Best Focus Sharpness for Various Search Methods


Conclusions

1. The variance measure is an effective sharpness measure that is well suited for autofocusing in the scanning electron microscope.

2. Autofocusing research has traditionally concentrated on fixed stepsize and iterative searches, but more sophisticated search algorithms can successfully reduce the total search time.

Documents

Sharpness Search Algorithms for Automatic Focusing in the …cbatten/pdfs/batten-image... · 2020. 11. 11. · Sharpness Search Algorithms for Automatic Focusing in the Scanning Electron