S. Mandayam, ECE Dept., Rowan University Automated Segmentation of Radiodense Tissue in Digitized Mammograms Department of Electrical & Computer Engineering

S. Mandayam, ECE Dept., Rowan University

Automated Segmentation of Automated Segmentation of Radiodense Tissue in Radiodense Tissue in

Digitized MammogramsDigitized Mammograms

Automated Segmentation of Automated Segmentation of Radiodense Tissue in Radiodense Tissue in

Digitized MammogramsDigitized Mammograms

Department of Electrical & Computer Engineering

201 Mullica Hill RoadGlassboro, NJ 08028, USA

engineering.rowan.edu

Shreekanth MandayamShreekanth Mandayam

GE John F. Welch Technology CenterBangalore, India

July 15, 2005


Breast CancerBreast CancerNew Cases of Cancer in the

U.S. (women)

Breast32.07%

Genital12.70%

Digestive System18.23%

Skin4.02%

Soft Tissue0.58%

Respiratory System12.69%

Bones0.17%

Oral/Phalanx1.44%

Other2.46%

Leukemia1.93%

Mulitple Myeloma1.03%

Urinary4.31%

Eye0.17%

Brain1.23%

Endocrine2.61%

Lymphoma4.36%


Survival RatesSurvival Rates

0 100%

I 98%

IIA 88%

IIB 76%

IIIA 56%

IIIB 49%

IV 16%

Each stage designates the size of the tumor how much it has spread.

Stage 0 Cancer:

Lobular Carcinoma in Situ (LCIS)

Ductal Carcinoma in Situ (DCIS)

20% of all diagnosed cancers


Risk Factor High-Risk Group Low-Risk Group Relative risk

Age Old Young > 4.0

Country of birth North America, Northern Europe

Asia, Africa > 4.0

Socioeconomic status High Low 2.0 – 4.0

Marital Status Never married Ever married 1.1 – 1.9

Place of residence Urban Rural 1.1-1.9

Place of residence Northern US Southern US 1.1-1.9

Race ≥ 45 years < 40 years

WhiteBlack

BlackWhite

1.1-1.91.1-1.9

Nulliparity Yes No 1.1-1.9

Age at first full-term pregnancy ≥ 30 years < 20 years 2.0-4.0

Age at menopause Late Early 1.1-1.9

Weight, postmenopausal women Heavy Thin 1.1-1.9

Any first-degree relative with history of breast cancer

Yes No 2.0-4.0

Mother and sister with history of breast cancer

Yes No > 4.0

Mammographic parenchymal patterns

Dysplastic Normal 4.0-6.0

Risk FactorsRisk Factors


Digitized MammogramDigitized Mammogram

Radiolucent

RadiodenseFilm region


Breast Density and Breast Cancer RiskBreast Density and Breast Cancer Risk

“……..women who had a breast density of 75% or greater had an almost fivefold increased risk of breast cancer…………”

– Byrne, C, et. al. “Mammographic features and breast cancer risk: effects with time, age, and menopause status,” Journal of the National Cancer Institute, Vol. 87, pp.1622-1629, 1995.


Breast Density and Breast Cancer RiskBreast Density and Breast Cancer Risk

“Women with extensive dense breast tissue visible on

mammogram have a risk of breast cancer that is 1.8 to 6.0 times that of women of the same age with little or no density.”

“…………….. the percentage of dense tissue on mammography at a given age has high heritability. Because mammographic density is associated with an increased risk of breast cancer, finding the genes responsible for this phenotype could be important for understanding the causes of the disease.”

– Boyd, N.F., et al, “Heritability of mammographic density, a risk factor for breast cancer,” New England Journal of Medicine, Volume 347(12), September 19, 2002, pp. 886-894.


Collaboration with Fox Chase Collaboration with Fox Chase Cancer Center, Phila.Cancer Center, Phila.

• Correlation between diet and breast cancer risk

• Two populations– FRAP (Family Risk Analysis Program):

Caucasian– Chinese-American: Three Generations

(Grandmothers, mothers, daughters)


TeamTeam

• Rowan University– Dr. Shreekanth Mandayam, Jeremy Neyhart, Rick

Eckert, Mike Kim, Maggie Kirlakovsky, Laura Coleman, Lyndsay Burd, Dan Barrot, Kevin Kanauss

• Fox Chase Cancer Center, Philadelphia– Dr. Marilyn Tseng, Dr. Kathy Evers MD

• Harvard Medical School/George Washington– Dr. Celia Byrne


Previous WorkPrevious Work

• Wolfe’s classification.

• “Toronto” method.

• Automated techniques.– “Main goal of research conducted at

Rowan University”


Wolfe’s ClassificationWolfe’s Classification

• N1: The breast is comprised entirely of fat.

• P1: The breast has up to 25% nodular densities.

• P2: The breast has over 25% nodular mammographic densities.

• DY: The breast contains extensive regions of homogeneous mammographic densities.


““Toronto” MethodToronto” Method

Display Results

33.3% RD

66.6% RL

Load Image into Computer

Set Boundary Threshold

1 4096

Set TissueThreshold

1 4096

Count pixels inregions

1 4096

Display Results

33.3% RD

66.6% RL

Load Image into Computer

Set Boundary Threshold

1 4096

Set TissueThreshold

1 4096

Count pixels inregions

1 4096


AutomatedAutomated

Proponents Approach Advantages Disadvantages

Lou and Fan [35] Adaptive fuzzy K-means technique to classify pixels as radiodense.

7.98 % error among 81 mammogram images.

18 seconds process time per image.

Zou et al. [36,37] Rule based histogram classifier

Maximum difference 20% from expert analysis.

No objective method for validation.

Bovis and Singh [38]

Classification using texture analysis.

91 % correct classification. Relies on knowledge of the region to be segmented.

Classifier is based on simplistic measures of texture.

Saha, Udupa, et al. [39]

Scale-based fuzzy connectedness

models

Estimates correlate strongly with analysis by radiologist.

Does not automatically exclude pectoral muscle.

Neyhart et al. [40]Eckert et al.

[41]

Constrained Neyman-Pearson decision

functionw/wo

Compression Adjustment

Automated technique Performance fit to database tested with. Weak inter-

dataset performance.


Limitations of Limitations of Previous MethodsPrevious Methods

• Highly qualitative and subjective• Requires user (radiologist’s) interaction• Do not provide relationship to actual breast

density measurements• Requires knowledge of region of interest

• No completely automated system exists – requires expert user interaction


Low Radio-density, Dark Image

High Radio-density, Dark Image

High Radio-density, Bright Image

Low Radio-density, Bright Image

Statement of the ProblemStatement of the Problem

Cannot use a single threshold for every image!!!!Radiodensity is not objectively defined……….


Mammography ProcedureMammography Procedure

Compression Plate

Compression Plate

Film Holder

Pectoral Muscle

Film Holder

MLOView

CCView


Overall Approach: FlowOverall Approach: Flow

Threshold Radiodense tissue quantified

X =

Original Image Mask Segmented Tissue

1 0

Gray-level

Dec

isio

n

Radiolucent

Radiodense


Image Model: Image Model: Gaussian Random FieldGaussian Random Field

),(),( yxwmyxf ff

Original Modeled


Baye’s ClassifierBaye’s Classifier

TB

Gray-level intensity

Num

ber

of P

ixel

s

Distribution 1(Radiolucent)

Distribution 2(Radiodense)

12, 21=2

2

221 BT


Neyman-Pearson ClassifierNeyman-Pearson Classifier



12, 21=2

2

12

212

2

NPT

1 2

TNP1 2

TNP


Num

ber

of P

ixel

s


DataDataBrigham and Women’s HospitalHarvard Medical SchoolCambridge, MA

Brigham and Women’s HospitalHarvard Medical SchoolCambridge, MA

Fox Chase Cancer CenterPhiladelphia, PAFox Chase Cancer CenterPhiladelphia, PA

Rowan UniversityGlassboro, NJRowan UniversityGlassboro, NJ

Digitized bmpImages (10)Digitized bmpImages (10)

Mammogram X-Ray Films

Digitized jpg & dicomImages (717)

FRAP (339) Chinese-American (378)

Digitizers:Agfa (OD = 3.0)Lumisys (OD = 3.85)



Constrained Neyman-PearsonConstrained Neyman-Pearson

TCNP


Num

ber

of P

ixel

s



12, 21=2

2

2212

221

CNPT


Test Images: Harvard-10Test Images: Harvard-10

19131709 20110811 11599502 15839502 11051702

26253102 28657701 14480101 27786202 26799401


CNP Results: Harvard-10CNP Results: Harvard-10


CNP Results: Harvard-10CNP Results: Harvard-10

0

10

20

30

40

50

60

70

1913

170

2011

081

1159

950

1583

950

1105

170

2625

310

2865

770

1448

010

2778

620

2679

940

Image Number

Per

cen

t R

adio

den

sity

"Toronto" Method Percentage Radiodensity

Constrained Neyman Pearson Percentage Radiodensity


Effect of Tissue CompressionEffect of Tissue Compression

Compression Plate

Film Holder

CCView

More StressHere

Less StressHere

More DensityHere

Less DensityHere


Parametric Model for Parametric Model for Tissue Location & CompressionTissue Location & Compression

2

2

2)( x

CNPv ekTxT

x

chestedgechestt ffNt

fy


Test Images: Harvard-10Test Images: Harvard-10

19131709 20110811 11599502 15839502 11051702

26253102 28657701 14480101 27786202 26799401


CNP-SV Results: Harvard-10CNP-SV Results: Harvard-10

19131709 20110811 11599502 15839502 11051702

26253102 28657701 14480101 27786202 26799401


CNP, CNP-SV and Toronto CNP, CNP-SV and Toronto Comparisons: Harvard-10Comparisons: Harvard-10

0

10

20

30

40

50

60

70

8019

1317

09

2011

0811

1159

9502

1583

9502

1105

1702

2625

3102

2865

7701

1448

0101

2778

6202

2679

9401

Image number

Per

cen

tag

e ra

dio

den

se t

issu

e

Percentage radiodense using Toronto method

Percentage radiodense using CNPA

Percentage radiodense using SVTA






Digitized bmpImages (10)Digitized bmpImages (10)







Methods StudiedMethods Studied

• Textures– Correlation Filters– Co-occurrence Matrix– Gabor Filters– Law’s Energy


Local Contrast EstimationLocal Contrast Estimation

• Maximize the local contrast between boundaries of connected tissue regions


Local Contrast Estimation (LCE)Local Contrast Estimation (LCE)

• Image Preprocessing• Tissue Segmentation

Mask• Compensation for

Compression• Threshold Selection

based on Local Contrast Estimation.

Tissue Segmentation

Compression Mask

Threshold Based on Global Estimate of Range

Image Compression Adjustment

Image Pre-Processing

Radiodensity Estimation of Mammogram


Image PreprocessingImage Preprocessing

Stripes caused during mammogram scan


Image ProcessingImage Processing

50 Percent of Left Side

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0

500

1000

1500

2000

2500

3000

3500

4000

4500

Mean.3715

1.5 SigmaThreshold

Statistical analysis

Binary Segmentation

Stripe Removal


Local Contrast Estimation (LCE)Local Contrast Estimation (LCE)

• Perceive connected regions a layers.


From the mask, the locations of these

artificial boundaries created by threshold

t is then found


Using threshold t, the mask of the

radiodense regions is created

Step 1: Estimation of the Boundaries


Local Contrast EstimationLocal Contrast EstimationStep 1: Boundary Estimation

Any white pixel in the new boundary mask will correspond a region where the estimated threshold t believes there is a change from radiolucent to radiodense tissue.


Local Contrast EstimationLocal Contrast EstimationStep 2: Calculation of Local Contrast

From this region selected by the boundary mask, a collection of pixels is gathered.


Split into two groups,

7 numbers higher than median

7 number lower than median

= local Edge Function

Local Contrast EstimationLocal Contrast EstimationStep 2: Calculation of Local Contrast

Calculate median

Find Difference of two group means

MH - ML


Local Contrast EstimationLocal Contrast EstimationStep 3: Calculation of Global Contrast

After the local contrast estimation is obtained for all regions defined by the mask, and average global estimate is obtained.

N

iContrastContrast

N

iLocal

Global

1

)(

)()()(

))(max(

LowH

Local

GroupmeanGroupmeaniContrast

iContrastContrast

N being the total number of Local Contrasts


• A sweep of thresholds is done for each image.

0.46 0.48 0.5 0.52 0.54 0.56 0.58 0.6 0.62 0.64 0.660.07

0.08

0.09

0.1

0.11

0.12

0.13

0.14

Local Contrast EstimationLocal Contrast EstimationStep 4: Calculation of Optimum Contrast

Glo

bal

Con

tras

t

Threshold

(Only small region is being shown in graph)

Based on graph, the threshold with the highest global contrast is chosen as the optimum threshold


ResultsResults

• Databases.

• Scanners.

• LCE

• LCE vs. CNP vs. SV-CNP.






Digitized bmpImages (10)







Scanner ComparisonsScanner Comparisons

AGFA Scanner Lumisys Scanner



• Image Preprocessing• Tissue Segmentation

Mask• Compensation for

Compression• Threshold Selection

based on Local Contrast Estimation.

Tissue Segmentation

Compression Mask

Threshold Based on Global Estimate of Range

Image Compression Adjustment

Image Pre-Processing

Radiodensity Estimation of Mammogram


Image PreprocessingImage Preprocessing


Tissue Mask SegmentationTissue Mask Segmentation


Tissue Mask SegmentationTissue Mask Segmentation


Compression Compensation MaskCompression Compensation Mask


Comparison between 3 methodsComparison between 3 methods


Problems with CNPProblems with CNP

2212

221

CNPT

Supervised Parameter


Problems with SV-CNPProblems with SV-CNP

• Based on threshold from CNP….

• Compression values over fit data to correlate with percentages.

• Final segmentation results do not visually match with the expected segmentation.

















Harvard ResultsHarvard Results

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

11051702 11599502 14480101 15839502 19131709 20110811 26253102 26799401 2778620 28657701

TorontoCNPSV-CNPLCE


Harvard ResultsHarvard Results

91% 1091

92% 495

87% 879

CNP

SV-CNP

LCE

MSECorrelation












34 selected validation images


CNP FCCCCNP FCCC

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

90.0Toronto

CNP


SV-CNP FCCCSV-CNP FCCC

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

90.0Toronto

SV-CNP


LCE FCCCLCE FCCC

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

90.0

Toronto LCE


FCCC ResultsFCCC Results

-39% 21732

48% 15127

85% 4052

CNP

SV-CNP

LCE

MSECorrelation












34 selected validation images


Combined ResultsCombined Results CNP SV-CNP LCE

Correlation compared to Toronto method (with

flagged) 0.147 0.565 0.851

Correlation compared to Toronto method (without flagged) 0.306 0.733 0.882

MSE compared to Toronto method (with

flagged) 22823.930 15622.682 4931.374

MSE compared to Toronto method (without flagged) 14381.540 5425.792 2811.690

Average % difference compared to Toronto method (with flagged) 18.186 12.924 8.232

Average % difference compared to Toronto

method (without flagged) 17.889 8.791 7.927


AnalysesAnalyses

• Correlation (Pearson r)– To determine if there is a significant relationship

between the LCE and Toronto Methods• Results revealed a r (30) = .851, p = .001.

• This means: The correlation between LCE and Toronto is .851, then 72.5% of the differences between LCE in terms of the relationship is predictable on the basis of the Toronto Method.

• 27.5% is not predictable, due to error.


AnalysesAnalyses

• Linear Regression– Since there was such a high correlation and strong

significance found, ran a linear regression to confirm the conclusions of the correlation and develop a regression equation.

• 1 Independent Variable (Predictor)-LCE• 1 Dependent Variable (Predicted)-Toronto

– Results yielded that LCE scores are significant predictors to the Toronto method

• Y = -7.523 E -3 + (1.030)(LCE Method)• F (1, 30) = 78.834, p = .001• Effect size, ^2 = .725 (from correlation).


Linear RelationshipLinear Relationship

Scattergram of LCE and Toronto Method

0

0.2

0.4

0.6

0.8

0 0.2 0.4 0.6 0.8 1

Toronto Method (Dr. Byrne)

Lo

cal C

on

trast

Esti

mate

s (

LC

E)


Analysis SummaryAnalysis Summary

• Linear Regression exhibits and verifies that the LCE method can predict what a radiologist would detect to a significant degree (73% accurate).

• There will always be some variance explained by error (the image was poor, the position of the breast was incorrect, etc.).













Database ResultsDatabase Results

182 27.5%

133 20%

87 13%

73 11%

50 7.5%

32 4.8%

10 1.5%

0 0%

0 0%

0 0%

Out of 660 images

0%-10%

10%-20%

20%-30%

30%-40%

40%-50%

50%-60%

60%-70%

70%-80%

80%-90%

90%-100%

# images % of database

14% of the images could not be evaluated


Database ResultsDatabase Results

37.9% 23.9%

19.5% 19.5%

9.8% 14%

10% 11%

5.7% 9.9%

1.4% 3.3%

.3% 1.5%

0% 0%

0% 0%

0% 0%

0%-10%

10%-20%

20%-30%

30%-40%

40%-50%

50%-60%

60%-70%

70%-80%

80%-90%

90%-100%

FRAP Chinese American


Database IssuesDatabase Issues

• 93 images could not be analyzed.


Summary of AccomplishmentsSummary of Accomplishments

• Development of a comprehensive database from multiple age and ethnic groups.

• Development of a completely automated radiodense tissue segmentation procedure.

• Comparison of new method with a previously established segmentation method.

• Algorithm has the ability to sift through entire databases of digitized mammograms quickly.


ConclusionsConclusions

• LCE is able to give good performances across multiple databases without the need to supervise.

• LCE is fully automated.• LCE is 86% correlated with an established

method• The average difference in percentage is less

than 8.3%


AcknowledgementsAcknowledgements

• “Dietary Patterns and Breast Density in Chinese-American Women,” American Cancer Society, Award Amount: $333,000; 2002– 2007.

• “Dietary Patterns and Breast Density,”American Institute of Cancer Research, Award Amount: $55,000; 2000 – 2002.

• “Digital Imaging Across the Curriculum,” National Science Foundation, Award Amount: $74,998, 2003-2005.


DiscussionDiscussion

Documents

S. Mandayam, ECE Dept., Rowan University Automated Segmentation of Radiodense Tissue in Digitized Mammograms Department of Electrical & Computer Engineering