Medical Background: Colon
Length: ~ 1.5 m Diameter: ~ 6 cm „tubelike“
Medical Background: Colon
Common diseases:
Colitis
Diverticulitis
Colon cancer
Colon cancer
70.000 cases/year (in Germany)
one of the leading causes of cancer death worldwide: 655.000 deaths/year
Colon polyps
Untreated polyps can develop into cancer.
Colonoscopy: cancer prevention (detect and remove polyps).
Problem: miss-rate (up to 25 %)
System for computer-assisted detection
Supports the doctor during examination
Unsolved problem
Many approaches
Lack of good data
General approach
Acquire data (videos / images) including ground truth information
Extract features
Train classifier
Test classifier
} find „the best“ features
1. Data acquisition
Videos: Capture colonoscopy in hospital
„Ground truth“: time consuming
2. Feature extraction
Divide image into patches
Extract features:TextureColor…
One feature-vector for each patch
3. Classifier training
Example: Support Vector Machine (SVM)
We have: set of feature vectors, each belonging to one class (polyp or non-polyp)
SVM: Hyperplane
4. Classifier testing
Test and training sets must be seperated! (e.g.: n-fold cross-validation)
Possible results for the patches:true positive (tp), false negative (fn)false positive (fp), true negative (tn)
Receiver Operation Characteristics (ROC) Graph
Ordinate: Abscissa:
Sensitivity= TPFNTP
Specificity= TNFPTN
Sensitivity 1−Specificity
Our approach
We have: 4 hours video of colonoscopy Full HD (1920 x 1080)
4 scenes, each showing a different polyp Varying distance, angle, illumination
Texture feature extraction:Grey-level co-occurrence Matrix (GLCM)Local binary pattern
Grey-level co-occurrence Matrix (GLCM)
GLCM Greyimage of size
Thus, is a matrix of size where is the number of possible grey-levels in
can be normalized by dividing each entry by
the sum of all entries (→ probabilties)
pi , j =∑n=1
N
∑m=1
M
{1 if I n ,m=i∧I n x ,m y= j0 else
pI N×M
p G×G GI
p
GLCM: example
100
22
0 1 0
Image
2
200
02
1 1 0
GLCM (not normalized)
0
The GLCM is parameterized by and Here: and
d x d yd x=1 d y=0
GLCM: statistical features
e.g. homogeneity:
Energy, correlation, inertia, …
These statistical features can form a feature-vector.
f homogeneity=∑i=0
G−1
∑j=0
G−1 p i , j 1i− j2
Local Binary Pattern (LBP)
1286432
168
1 2 4
Weights
2367
1812
21 4 3
Example neighbourhood
9
010
11
1 0 01
+ 8+ 16+ 64
LBP = 89
LBP
LBP-value (computed from each neighbourhood)
All LBP-values form a histogram that can be used as a feature-vector
3×3
Extension: Opponent-colour LBP
One LBP-histogram from each color-channel
Additionally: Intra-channel histograms:center-pixel and neighbourhood from different
color-channels
In total: 9 histograms form the feature-vector → many dimensions
Experiments
Data: 4 scenes
Feature-extraction: 4 different featuresetsGLCM 6GLCM 16LBPOC-LBP
4 different patch-sizes
Classifier-training and testing (LibSVM) Stratified 4-fold cross-validation
ROC-graph (example)
Results (AUC)
Scene Patchsize GLCM6 GLCM16 LBPOC-LPB
1 70 0.83 0.95 0.89 0.94
2 70 0.70 0.75 0.76 0.87
3 70 0.89 0.88 0.95 0.96
4 70 0.65 0.68 0.80 0.91
1 50 0.80 0.89 0.86 0.89
2 50 0.71 0.75 0.71 0.84
3 50 0.80 0.83 0.88 0.95
4 50 0.63 0.65 0.77 0.89
1 35 0.77 0.92 0.84 -
2 35 0.65 0.71 0.66 -
3 35 0.74 0.76 0.82 -
4 35 0.63 0.65 0.74 -
Results
OC-LPB almost always the best GLCM6 almost always the worst
GLCM performs worse on scene 4 LPB performs worse on scene 2
Independent from the features:Scene 1 and 3: good resultsScene 2 and 4: worse results
No relation between feature and „polyp- types“
Future Work
More video/image data
Method for ground truth aqcuisition
Test / develop more features
Realtime
References
GeneralAMELING, S.: Polypen- und Tumordetektion in Koloskopie-Videos, Studienarbeit im Studiengang
Computervisualistik, Universität Koblenz-Landau, 2008
Miss-ratesBRESSLER, Brian ; PASZAT, Lawrence F. ; VINDEN, Christopher ; LI, Cindy; HE, Jingsong ; RABENECK,
Linda: Colonoscopic miss rates for right sided colon cancer: a population-based analysis. In: Gastroenterology 127 (2004), Nr. 2, S. 452–456
THOMSON, Alan ; AHNEN, Dennis ; RIOPELLE, John: Intestinal polypoid adenomas. In: eMedicine, The Continually Updated Clinical Reference (2007)
Polyp Detection MethodsIAKOVIDIS, D.K. ; MAROULIS, D.E. ; KARKANIS, S. A.: An intelligent system for automatic detection of
gastrointestinal adenomas in video endoscopy. In: Computers in Biology and Medicine 36 (2006), Nr. 10, S. 1084–1103
Local Binary PatternsMäenpää, T.: The local binary pattern approach to texture analysis–extensions and applications. (2003)
Dissertation, University of Oulu.
Grey-level co-occurrence MatrixHARALICK, R. M. ; DINSTEIN, I. ; SHANMUGAM, K.: Textural features for image classification. In: IEEE
Trans. Systems, Man, and Cybernetics 3 (1973), Nr. 6, S. 610–621