SEEING IS BELIEVING?
Chris Glasbey
Biomathematics and Statistics Scotland
Image analysis is the extraction of information from pictures
Which line is the longer? Who is this?
2
The human eye/brain is a superb image analyser
So, why use a computer?
• better for quantitative tasks (repeatable/non-subjective)
• cheaper, faster and less tedious
• capable of different techniques
3
An example of non-automated image analysis:
Cross-section of turbinate bone
=⇒Using sno-pake and black ink.
4
But automating image analysis is hard, because all a computer ‘sees’ is:
132 128 126 123 137 129 130 145 158 170 172 161 153 158 162 172 159 152139 136 127 125 129 134 143 147 150 146 157 157 158 166 171 163 154 144144 135 125 119 124 135 121 62 29 16 20 47 89 151 162 158 152 137146 132 125 125 132 89 17 19 11 8 6 9 17 38 134 164 155 143142 130 124 130 119 15 46 82 54 25 6 6 11 17 33 155 173 156134 132 138 148 47 92 208 227 181 111 33 9 6 14 16 70 180 178151 139 158 117 22 162 242 248 225 153 62 19 8 8 11 13 159 152153 135 157 46 39 174 207 210 205 136 89 52 17 7 6 6 70 108167 168 128 17 63 169 196 211 168 137 121 88 21 9 7 5 34 57166 170 93 16 34 63 77 140 28 48 31 25 17 10 9 8 22 36136 111 83 15 48 69 57 124 55 86 52 112 34 11 9 6 15 3049 39 46 11 83 174 150 128 103 199 194 108 23 12 12 10 14 3426 24 18 14 53 175 153 134 98 172 146 59 13 14 13 12 12 4621 16 11 14 21 110 126 47 62 142 85 33 10 13 13 11 11 1517 14 10 11 11 69 102 42 39 74 71 28 9 13 12 12 11 1818 19 11 12 8 43 126 69 49 77 46 17 7 14 12 11 12 1924 30 17 11 12 6 73 165 79 37 15 12 10 12 13 10 10 1624 40 18 9 9 2 2 23 16 10 9 10 10 11 9 8 6 1043 40 25 6 10 2 0 6 20 8 10 16 18 10 4 3 5 739 34 23 5 7 3 2 6 77 39 25 31 36 11 2 2 5 217 16 9 4 6 5 6 36 85 82 68 75 72 27 5 7 8 04 8 5 6 8 15 65 127 135 108 120 131 101 47 6 11 7 42 9 6 6 7 74 144 170 175 149 162 153 110 48 11 12 3 5
11 9 3 7 21 127 176 190 169 166 182 158 118 44 10 11 2 58 0 5 23 63 162 185 191 186 181 188 156 117 38 11 12 25 333 5 6 64 147 182 173 190 221 212 205 181 110 33 19 42 57 505 3 7 45 160 190 149 200 253 255 239 210 115 46 30 25 9 59 4 10 16 24 63 93 187 223 237 209 124 36 17 4 3 2 17 8 13 8 9 12 17 19 26 41 42 24 11 5 0 1 7 4
Could you recognise Mona Lisa from this?
5
What are the data? microscopy medical scanner remote sensing
What is image size? 20µm 50cm 15km
What is measured? transmitted light X-ray absorption reflected light
How is objectsampled? focal plane cross-section perspective view
6
Three questions: 1. What is the 3-D shape of this diatom?
increasing focal plane
7
Question 2: How fat is this sheep?
⇒X-ray CT
SAC-BioSS CT Unit, led by Geoff Simm
8
Question 3: Can we distinguish between fish species?
haddock whiting
9
PLAN
0. Introduction
1. Microscopy
2. Medical scanners
3. Remote sensing (and fish)
10
1. Microscopy
Using computers, we can
• Visualise images in ways not possible optically
11
We can superimpose different microscope images of algae and bacteria
brightfield DIC
+
12
DIC fluorescent
+
(Nick Martin, SAC)
13
1. Microscopy
Using computers, we can
• Visualise images in ways not possible optically
• Count and measure more easily
14
1. Microscopy
Using computers, we can
• Visualise images in ways not possible optically
• Count and measure more easily
However, identifying objects to count and measure is more challenging if
• Objects vary in appearance
15
DIC image of yeast cells – example of template construction
↓ ↑
→ →integrate rotate differentiate
16
1. Microscopy
Using computers, we can
• Visualise images in ways not possible optically
• Count and measure more easily
However, identifying objects to count and measure is more challenging if
• Objects vary in appearance
• Optical effects distort what is seen
17
Brightfield image of cashmere fibres
These fibres are the same thickness, but look different
(Angus Russel, Macaulay Land Use Research Institute)
18
We use a mathematical model to understand how a fibre appears indifferent focal planes
→fibre oblique to focal plane
And use it to adjust for errors in measuring fibre thicknesses in images
19
1. Microscopy
Using computers, we can
• Visualise images in ways not possible optically
• Count and measure more easily
However, identifying objects to count and measure is more challenging if
• Objects vary in appearance
• Optical effects distort what is seen
Though optical distortions are sometimes an advantage
20
In answer to question 1: we can reconstruct the 3-D shape of this diatom
Brightfield images at series of focal planes
(Ed Breen, CSIRO – Mathematical and Information Sciences)
21
Step 1: compute local gradients in intensity
gradient: small large
22
Step 2: smooth the gradients
23
Step 3: at each location, choose image with maximum gradient
24
Step 4: reconstruct the 3-D surface
25
PLAN
0. Introduction
1. Microscopy
2. Medical scanners
3. Remote sensing (and fish)
26
⇒Ultrasound
(Geoff Simm, SAC)
27
To measure fat depth, we need to draw the red lines
28
We consider the pattern of pixel values on either side of a boundary, averagedover many images.
29
30
We compare the patterns with the different possible locations of a boundary,where darker values denote better fits:
top boundary bottom boundary
The best boundary is the darkest path between the two sides of each image
31
Illustration of Dynamic Programming:
Find connected path from left to right with minimum cost
32
One of 700 possible paths
33
Minimum cost paths to column 2
34
Minimum cost paths to column 3
35
All minimum cost paths
36
Minimum cost in final column
37
Trace path back to first column
38
Minimum cost path
39
Here we see the resulting automatic, and hand-drawn, boundaries:
40
In CT, pixel values range from −1000 to > +1000 Hounsfield units.
According to how we display these values we obtain different images.
41
For segmentation, we first ‘unroll’ the images
polar transformation boundary pattern
42
We then use level of X-ray attenuation to identify tissues
⇒automatic hand-drawn fat muscle bone
In answer to question 2: we predict sheep fatness from these measured areas
43
With a 3-D scanner there is more information but segmentation is harder
sheep skeleton leg muscles in pelvic region
44
PLAN
0. Introduction
1. Microscopy
2. Medical scanners
3. Remote sensing (and fish)
45
Landsat image of St Andrews, Scotland
It is rare to get such a cloud-free scene in Britain. Venus is even cloudier.
46
=⇒NASA Magellan mission SAR image of Mona Lisa crater, Venus
47
SAR (synthetic aperture radar) is also used on Earth
SAR image, East Anglia Map of field and road boundaries
(European Space Agency)
48
How the SAR image was obtained
x1
x2
h( , )x1 x2
φ1
f1
f2
We need to warp the SAR image to align with the map
49
Arad et al. (CVGIP: Graphical Models and Image Processing, 1994)
↘‖
→ →Venus Venus warped average
50
edges in SAR image
+
after warping
51
Estimated elevations
52
Resulting alignment
53
We can also use warping to identify fish
haddock 1 whiting 1
haddock 2 whiting 2
(Norval Strachan, Torry Research Centre)
54
Fish warping has a long history....
D’Arcy Thompson, On Growth and Form (1917)
55
haddock 1 haddock 2
↓ ‖
→warping haddock 1 ‘impersonating’
haddock 2
Dissimilarity between fish = difference after warp + distortion in warping
56
haddock 1 whiting 1
↓ ‖
→warping haddock 1 ‘impersonating’
whiting 1
57
average haddock average whiting
In answer to question 3: we can distinguish between fish species
Haddock are more similar to the average haddock
Whiting are more similar to the average whiting
58
Summary
Automated image analysis is a challenging problem,but offers the possibility of
• more accurate,
• faster,
• more sophisticated, measurements.
For further details, see
Glasbey, C.A. and Horgan, G.W. (1995)Image Analysis for the Biological Sciences. Wiley: Chichester.
and research papers on
http://www.bioss.ac.uk/~chris
59