Melanoma and skin cancers vs Image Processing. 2 Skin cancer and melanoma Skin cancer : most common of all cancers Skin cancer : most common of all cancers

Melanoma and skin Melanoma and skin cancers vs Image cancers vs Image

ProcessingProcessing

22

Skin cancer Skin cancer and melanomaand melanoma

Skin cancer : most common of all cancersSkin cancer : most common of all cancers

1.1. According to the latest statistics available from According to the latest statistics available from the National Cancer Institute,the National Cancer Institute, skin cancer is skin cancer is the most common of all cancers in the United the most common of all cancers in the United states. states.

2.2. More than 1 million cases of skin cancer are More than 1 million cases of skin cancer are diagnosed in the US each year.diagnosed in the US each year.

3.3. What’s shown here are some examples of skin What’s shown here are some examples of skin lesion images. lesion images.

4.4. The four images shown on the left are various The four images shown on the left are various form of skin lesions, cancerous or non-form of skin lesions, cancerous or non-cancerous. cancerous.

5.5. The two on the right are a specific form of skin The two on the right are a specific form of skin cancer: melanoma.cancer: melanoma.

33

What is Melanoma?What is Melanoma?1.1. A type of skin cancer A type of skin cancer

that starts from that starts from melanocytesmelanocytes

2.2. 66thth leading cause of leading cause of cancer death in the cancer death in the USUS

3.3. No single etiologyNo single etiology4.4. Some risk factors Some risk factors

include:include:1.1. Sun exposure -Sun exposure -

depleting ozone layer depleting ozone layer 2.2. Presence of many or Presence of many or

unusual molesunusual moles3.3. Skin typesSkin types4.4. Genetics Genetics

predispositionpredisposition

benign malignant

skin

66

Skin cancer and melanomaSkin cancer and melanoma Skin cancer : most common of all Skin cancer : most common of all

cancerscancers

[ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ]

Use of color to distinguish Use of color to distinguish malignant and benign tumors malignant and benign tumors

Skin tumors can be either malignant or benign

Clinically difficult to differentiate the early stage of malignant melanoma and benign tumors due to the similarity in appearance

Proper identification and classification of malignant melanoma is considered as the top priority because of cost function

Classification of skin tumors using computer imaging and pattern recognition

1. Previous texture feature algorithms successfully differentiate the deadly melanoma and benign tumor seborrhea kurtosis

2. Relative color feature algorithm is explored in this research for differentiate melanoma and benign tumors, dysplastic nevi and nevus

Successfully classify 86% of malignant melanoma using relative color features, compared to the clinical accuracy by dermatologists in detection of melanoma of approximately 75%

Types of MelanomaTypes of Melanoma Superficial Spreading MelanomaSuperficial Spreading Melanoma

7070%, neck, legs, pelvis %, neck, legs, pelvis Nodular Melanoma Nodular Melanoma

15%, dome-shaped nodule 15%, dome-shaped nodule Acral-Lentiginous MelanomaAcral-Lentiginous Melanoma

8 %, Common in dark-skin8 %, Common in dark-skin Lentigo Maligna Melanoma Lentigo Maligna Melanoma

5 %, sun-exposed area, mistaken for age spot5 %, sun-exposed area, mistaken for age spot Amelanotic Melanoma Amelanotic Melanoma

0.3%, non-pigmented0.3%, non-pigmented Desmoplastic Desmoplastic

1.7%, ½ amelanotic1.7%, ½ amelanotic

Benign vs Benign vs MalignantMalignant

99

1010

Automated Melanoma Recognition Automated Melanoma Recognition UsingUsing

Imaging Techniques Imaging Techniques

Melanoma is one of the most aggressive cancers, but it can be healed by surgical excision successfully only if it is recognized in the early stage.

Since the melanoma emerges as a tiny dot in the topmost skin layer, it can be examined during routine medical check up.

Although the lesions are accessible, in many cases, it is a difficult task to make decisions whether nevi are benign or malignant.

Further, frequent use of biopsy is also not encouraged.

Hence, to assist dermatologist's diagnosis, it is useful to develop an automated imaging-based melanoma recognition system.

1111

1.1. Uncontrolled growth of melanocytes give rise to Uncontrolled growth of melanocytes give rise to dark and elevated appearance of melanoma. dark and elevated appearance of melanoma.

2.2. Neoplasm- growth of tissue, tumorNeoplasm- growth of tissue, tumor

3.3. Melanoma is a type of malignant skin cancer that Melanoma is a type of malignant skin cancer that starts from melanocytes. It’s caused by starts from melanocytes. It’s caused by uncontrolled growth of melanocytes that gives rise uncontrolled growth of melanocytes that gives rise to tumor.to tumor.

1.1. Nonetheless there are Nonetheless there are risksrisks factors that highly factors that highly attributed to its incidence. Some of the them are:attributed to its incidence. Some of the them are:1.1. amount of sun exposure – the more cumulative amount of sun exposure – the more cumulative

exposure the higherexposure the higher

2.2. presence of many of unusual mole – people with many presence of many of unusual mole – people with many moles in the body moles in the body

3.3. Fitzpatrick’s Skin Type I and II have higher risk –1975 Fitzpatrick’s Skin Type I and II have higher risk –1975 Thomas Fitzpatrick, Harvard skin typing system based Thomas Fitzpatrick, Harvard skin typing system based on skin complexion and response to sun exposureon skin complexion and response to sun exposure

4.4. genetic predisposition – if there history of melanoma genetic predisposition – if there history of melanoma that runs in the familythat runs in the family

2.2. According to a study ,compared to general According to a study ,compared to general population, people who with 2 risk factors have population, people who with 2 risk factors have 3.5 times risk of developing MM and 20 times 3.5 times risk of developing MM and 20 times those who have 3 or more risk factors.those who have 3 or more risk factors.

1.1. These are the types of melanonaThese are the types of melanona

2.2. As you see, SSM is the most prevalent one As you see, SSM is the most prevalent one that makes up 70% of most diagnosed that makes up 70% of most diagnosed melanomamelanoma

3.3. In this work, images of superficial In this work, images of superficial spreading melanoma were only explored. spreading melanoma were only explored.

4.4. The reason being, and the problem that The reason being, and the problem that this work is trying to solve, Dysplastic Nevi this work is trying to solve, Dysplastic Nevi ( a benign mole) has properties that are ( a benign mole) has properties that are highly similar to this SSM melanoma, which highly similar to this SSM melanoma, which makes the diagnosis of melanoma difficult.makes the diagnosis of melanoma difficult.

Melanoma IncidenceMelanoma IncidenceAge Adjusted

All Ages, Both Sexes

0

5

10

15

20

25

30

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

Year

Rat

e p

er 1

00,0

00

Cauc

Af Am.

Asian

Incidence highest in Caucasian skin

Graph one- Caucasian has the highest incidence of MM. Having fair complexion is one of the risk factors. Researches attribute this to low level of melanin that absorbs harmful UV radiation in fair skin, thus UV penetrates much deeper layer affects the surrounding cells.

NCHS – national center for health statisticsBureau of Health Statistics

1.1. Graph two – men shows higher incidence than women. Graph two – men shows higher incidence than women.

2.2. A study of in Germany linked this trend to mutation of A study of in Germany linked this trend to mutation of genes called BRAF 4% and CDKN2A 1%.genes called BRAF 4% and CDKN2A 1%.

Age-AdjustedAll Ages, All Races

0

10

20

30

40

5019

92

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

Year

Rat

e p

er 1

00,0

00

Male

Female

Melanoma IncidenceMelanoma Incidence

Age-AdjustedAll Races, Both Sexes

0

10

20

30

40

50

60

70

80

90

1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Year

Rat

e p

er 1

00,0

00

<20

20-49

50-64

65-74

>74

Incidence increases with age

Graph thee – Incidence increases with age. Link to cumulative sun exposure

Some studies suggested that people who had significant exposure to UV at younger age have higher risk in later age when UV exposure decreases.

1.1. Age-adjusted- distribution of age by percentageAge-adjusted- distribution of age by percentage

2.2. It’s a way of data normalization so that you can It’s a way of data normalization so that you can compare two different countries, cities and so forth compare two different countries, cities and so forth

3.3. Need standard population distributionNeed standard population distribution

4.4. Who use itWho use it NCHS – national center for health statisticsNCHS – national center for health statistics Bureau of Health StatisticsBureau of Health Statistics

5.5. What to sayWhat to say1.1. So these are three graphs that show melanoma incidence in So these are three graphs that show melanoma incidence in

different dimensions: based on race, gender, and age. different dimensions: based on race, gender, and age.

2.2. Here, it’s evident that Melanoma has its favorites, so to speak.Here, it’s evident that Melanoma has its favorites, so to speak.

Melanoma IncidenceMelanoma Incidence

It is estimated that 62,480 men and women (34,950 men and 27,530 women) will be diagnosed with and 8,420 men and women will die of melanoma of the skin in 2008 (SEER)

Percent IncreaseSEER US Population Melanoma Incidence

Age-Adjusted

0

2

4

6

8

10

12

14

1975

1977

1979

1981

1983

1985

1987

1989

1991

1993

1995

1997

1999

2001

2003

2005

Year

Per

cen

t (%

)

Surveillance Epidemiology and End Surveillance Epidemiology and End Results Results

What to sayWhat to say This is the combination of all of the This is the combination of all of the

data from the previous slides. data from the previous slides. Average of 4.2 percent increase per Average of 4.2 percent increase per

yearyear

Survival Rate by StageSurvival Rate by StageThe American Joint Committee on Cancer (AJCC) TNM System

5- and 10-Year Survival Rate40,0000 between 1988-2001

0

20

40

60

80

100

120

I IA IIA IIB IIC IIIA IIIB IIIC IV

Melanoma Stage

Per

cen

t (%

)

5 Year

10 Year

http://www.cancer.org

1.1. The imaging is performed by a special CCD The imaging is performed by a special CCD camera combined with an epiluminescence camera combined with an epiluminescence microscope in order to produce digitalized ELM microscope in order to produce digitalized ELM images of the skin lesions. images of the skin lesions.

2.2. Once the images are captured, the lesion has Once the images are captured, the lesion has to be segmented from the background and to be segmented from the background and useful information should be extracted from useful information should be extracted from the lesion region. the lesion region.

3.3. Based on the extracted features, decisions Based on the extracted features, decisions have to be made about the nature of the skin have to be made about the nature of the skin lesion. lesion.

4.4. The decisions should be supported by The decisions should be supported by descriptive justifications so that dermatologist descriptive justifications so that dermatologist can understand the decision making process. can understand the decision making process.

2222

Contact person: Assoc. Prof. Contact person: Assoc. Prof. PonnuthuraiNagaratnamSuganthan, email: PonnuthuraiNagaratnamSuganthan, email: [email protected]: [email protected]: 6790-5404

Collaborators: Prof. C L Goh, MD, National Skin Centre, Collaborators: Prof. C L Goh, MD, National Skin Centre, Singapore & Dr. H Kittler, University of Vienna Singapore & Dr. H Kittler, University of Vienna

This is an on-going project. We have implemented the This is an on-going project. We have implemented the segmentation, feature extraction and clasifcation modules segmentation, feature extraction and clasifcation modules satisfactorily, although further improvements are satisfactorily, although further improvements are desirable. The module to provide explanations supporting desirable. The module to provide explanations supporting the classifcation decisons is yet to be developed. siiithe classifcation decisons is yet to be developed. siii

2323

2424

Skin cancer and Skin cancer and melanomamelanoma

Skin cancer : most common of all cancersSkin cancer : most common of all cancers Melanoma : leading cause of mortality (75%)Melanoma : leading cause of mortality (75%)


1. Although represent only 4 percent of all skin cancers in the US, melanoma is the leading cause of mortality.

2. They account for more than 75 percent of all skin cancer deaths.

2525

Skin cancer Skin cancer and and

melanomamelanoma

Skin cancer : most common of all cancersSkin cancer : most common of all cancers Melanoma : leading cause of mortality (75%)Melanoma : leading cause of mortality (75%) Early detection significantly reduces mortalityEarly detection significantly reduces mortality


The time line shown here is the 10 year survival rate of melanoma. 1.If caught in its early stage, as seen here, melanoma can often be cured with a simple excision, so the patient have a high chance to recover. Hence, early detection of malignant melanoma significantly reduces mortality.

2626[ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ]

Clinical ViewClinical ViewDermoscopy view

DermoscopyDermoscopy1.1. Dermoscopy is a noninvasive imaging technique, and Dermoscopy is a noninvasive imaging technique, and

it is just the right technique for this task.it is just the right technique for this task.

2.2. It has been shown effective for early detection of It has been shown effective for early detection of melanoma. melanoma.

3.3. The procedure involves using an incident light The procedure involves using an incident light magnification system, i.e. a dermatoscope, to magnification system, i.e. a dermatoscope, to examine skin lesions. examine skin lesions.

4.4. Often oil is applied at the skin-microscope interface. Often oil is applied at the skin-microscope interface.

5.5. This allows the incident light to penetrate the top This allows the incident light to penetrate the top layer of the skin tissue and reveal the pigmented layer of the skin tissue and reveal the pigmented structures beyond what would be visible by naked structures beyond what would be visible by naked eyes. eyes. 2727

2828

DermoscopyDermoscopy Dermoscopy improves diagnostic accuracy Dermoscopy improves diagnostic accuracy

by 30% in the hands of trained physiciansby 30% in the hands of trained physicians May require as much as 5 year experience to May require as much as 5 year experience to

have the necessary traininghave the necessary training Motivation for Computer-aided diagnosis Motivation for Computer-aided diagnosis

(CAD) of pigmented skin lesion from these (CAD) of pigmented skin lesion from these dermoscopy images. dermoscopy images.

Clinical view Dermoscopy view

In the future, with the In the future, with the development of new development of new algorithms and techniques, algorithms and techniques, these computer procedures these computer procedures may aid the dermatologists to may aid the dermatologists to bring medical break through in bring medical break through in early detection of melanoma. early detection of melanoma.

2929

40,000 people between 1988-200140,000 people between 1988-2001 Cancer stage is categorized into TNM levelCancer stage is categorized into TNM level T – tumor ( localized)T – tumor ( localized) N – regional lymph-nodesN – regional lymph-nodes M -Metastasis M -Metastasis The key point is the earlier the better of survivalThe key point is the earlier the better of survival 5- and 10- year survival mean percentage of 5- and 10- year survival mean percentage of

people who live at least 5 and 10 years people who live at least 5 and 10 years respectively after being diagnosedrespectively after being diagnosed

DiagnosiDiagnosis- s- ABCDABCDEE SystemSystem

1. E= evolution/elevation2. What to say3. ABCDE system is the tool for detecting melanoma.

This is a list of criteria that can be used for distinguishing between benign and malignant melanocytic skin lesions.

4. A- if you draw a line across the center of MM, you’ll see that is not symmetric compared to regular mole

5. B- the border is uneven or ragged is a sign of melanoma

6. C-if there are multiple shades of pigment is presence

7. D- diameter > 6mm8. Dermatologist adds E for either evolution if lesion

changes upon observation or E for elevation.

9. Suspicious lesion is followed by histological confirmation.

Where the problems lieWhere the problems lie Atypical nevi acquire several properties similar to Atypical nevi acquire several properties similar to

melanoma, their recognition posed high difficulties melanoma, their recognition posed high difficulties even to experts. The classical ABCD guidance is not even to experts. The classical ABCD guidance is not reliable therefore cannot be used as sole indicator reliable therefore cannot be used as sole indicator for detection of melanoma for both clinical and for detection of melanoma for both clinical and public examination. public examination.

In clinical setting, recognition and discrimination In clinical setting, recognition and discrimination are highly subjective with rate of success based on are highly subjective with rate of success based on experts’ years of experience. As was found, experts’ years of experience. As was found, inexperienced dermatologists showed decrease inexperienced dermatologists showed decrease sensitivity in the detection of melanoma in both live sensitivity in the detection of melanoma in both live and photo examinations.and photo examinations.

General practitioner – 62% sensitivity and 63% specificity General practitioner – 62% sensitivity and 63% specificity Dermatologist – 80% sensitivity and 60% specificityDermatologist – 80% sensitivity and 60% specificity

OK, so we have the ABCD diagnosis tool plus the experts. OK, so we have the ABCD diagnosis tool plus the experts. So anyone with sort of skin lesion can step in a clinic get So anyone with sort of skin lesion can step in a clinic get

the ABCD tool and experts examination undertaken then the ABCD tool and experts examination undertaken then there you have the results. there you have the results.

You either have benign mole or malignant melanoma at You either have benign mole or malignant melanoma at the end of the consultation. Everything just goes as plan.the end of the consultation. Everything just goes as plan.

Unfortunately it is not always the case.Unfortunately it is not always the case. Sensitivity – TP/TP+FNSensitivity – TP/TP+FN Specificity – TN/TN+FPSpecificity – TN/TN+FP

Read the bulletRead the bullet The objective of the this work is to address these The objective of the this work is to address these

problems problems

MM and DNMM and DNABCD RulesABCD Rules

Malignant Melanoma

Dysplastic Nevi

Here you have some samples of MM on the top row and DN on the bottom rowAtypical Nevi (mole) – shares some sometimes all characteristics of MM.

This actually what makes melanoma detection difficult.

ObjectivesObjectivesTo construct an automated, image-based system for To construct an automated, image-based system for

classification of Malignant Melanoma and Dysplastic classification of Malignant Melanoma and Dysplastic Nevi using solely the visual texture information of Nevi using solely the visual texture information of the lesion. The system will be based on the lesion. The system will be based on methodologies that emanate and/or correlated with methodologies that emanate and/or correlated with human vision therefore will closely emulates human human vision therefore will closely emulates human experts only with greater extent of accuracy, experts only with greater extent of accuracy, reliability and reproducibilityreliability and reproducibility Investigate new segmentation methods that will Investigate new segmentation methods that will

be effective on both lesionsbe effective on both lesions Extract most relevant texture information from Extract most relevant texture information from

the imagethe image Construct a classification system of the lesionConstruct a classification system of the lesion

1.1. Ultimate goal is the construction Ultimate goal is the construction of the classification systemof the classification system

2.2. The uniqueness of the system is the fact The uniqueness of the system is the fact that :that :

only texture information is used – robust in only texture information is used – robust in color variabilitycolor variability

Methodologies used through out the whole Methodologies used through out the whole process emanate from the human vision thus process emanate from the human vision thus emulate human expertemulate human expert

Systems, Materials and Systems, Materials and ToolsTools

Image databaseImage database Original tumor imagesOriginal tumor images

512x512 24-bit color images digitized from 35mm color 512x512 24-bit color images digitized from 35mm color photographic slides and photographsphotographic slides and photographs

160 melanoma, 42 dysplastic, and 80 nevus skin tumor 160 melanoma, 42 dysplastic, and 80 nevus skin tumor imagesimages

Border imagesBorder images Binary images drawn manually and reviewed by the Binary images drawn manually and reviewed by the

dermatologist for accuracy dermatologist for accuracy SoftwareSoftware

CVIPtoolsCVIPtools Computer vision and image processing tools developed Computer vision and image processing tools developed

at our research labat our research lab PartekPartek

Statistical analysis toolsStatistical analysis tools

CVIPtoolsCVIPtools

Other approach - Other approach - TextureTexture

System for System for Melanoma DetectionMelanoma Detection

Outline of the ProcessOutline of the Process1. Here you have the

outline of the process2. Each of the subsequent

step is dependent of the of the preceding steps. In other terms, the results of subsequent step is only as good as the results of preceding steps.

3. Therefore, since segmentation is the top most of the hierarchy, its important to make sure the method is robust.

HypothesesHypotheses Due to observable pattern disruption in the skin Due to observable pattern disruption in the skin

tissue driven by the MM, It is hypothesize that tissue driven by the MM, It is hypothesize that measuring magnitude of pattern disruption measuring magnitude of pattern disruption provides discriminative features for diagnosing provides discriminative features for diagnosing MM. MM.

Since visual texture is highly length-scale Since visual texture is highly length-scale dependent, It is hypothesized that the detection dependent, It is hypothesized that the detection and analysis methods that explore texture at and analysis methods that explore texture at different scales such as the wavelet is the most different scales such as the wavelet is the most appropriate approach. appropriate approach.

It is hypothesized that texture descriptors that It is hypothesized that texture descriptors that emanate from and highly correlated with human emanate from and highly correlated with human vision system provide the utmost representation, vision system provide the utmost representation, and thus yield a more contextual system—a and thus yield a more contextual system—a system that closely emulate human expert system that closely emulate human expert

Item one – skin has distinct uniform pattern (glyphic pattern). Item one – skin has distinct uniform pattern (glyphic pattern). MM disrupts texture. MM disrupts texture. Quantifying texture differences between MM and NV is more Quantifying texture differences between MM and NV is more

reliable method than color-based ( color-based in prone to reliable method than color-based ( color-based in prone to variability in imaging system)variability in imaging system)

Item two – texture come in different sizes. Item two – texture come in different sizes. Detection method that explore texture image at different Detection method that explore texture image at different

possible scale is more sensitive than methods that are using possible scale is more sensitive than methods that are using one scale. one scale.

Example of this snake-based ( gradient-based), Normalized Cut, Example of this snake-based ( gradient-based), Normalized Cut, histogram thresholdhistogram threshold

Item three – there are many texture descriptors that Item three – there are many texture descriptors that are purely algorithmic that may not necessary are purely algorithmic that may not necessary correlate with human vision. correlate with human vision.

One example is first-order statistics of texture One example is first-order statistics of texture ( variance ,mean), structure-based approach, ( variance ,mean), structure-based approach, laplacain of Gaussian. laplacain of Gaussian.

Texture classifiers that emanate from or highly Texture classifiers that emanate from or highly correlated with human visual system provides a closer correlated with human visual system provides a closer approximation of experts perception of texture. approximation of experts perception of texture.

Visual Visual TexturTextur

ee

TextureTexture

Texture is regarded as what constitutes a macroscopic region. Its structure is Texture is regarded as what constitutes a macroscopic region. Its structure is simply attributed to the simply attributed to the repetitive patterns in which elements or primitives are repetitive patterns in which elements or primitives are arranged according to a placementarranged according to a placement rule(Tamura et al, 1978). rule(Tamura et al, 1978).

Texture is both the Texture is both the number and types of its (tonal) primitive and their spatial number and types of its (tonal) primitive and their spatial arrangementarrangement (Haralick ,1979). (Haralick ,1979).

The term texture generally refers to The term texture generally refers to repetition of basic texture elements called repetition of basic texture elements called texelstexels. The texel contains several pixels, whose placement could be periodic, . The texel contains several pixels, whose placement could be periodic, quasi-periodic, or random. Natural textures are generally random, whereas quasi-periodic, or random. Natural textures are generally random, whereas artificial textures are often deterministic or periodic. Texture may be course, artificial textures are often deterministic or periodic. Texture may be course, fine, smooth, granulated, rippled, regular, irregular, or linear (Jain, 1989).fine, smooth, granulated, rippled, regular, irregular, or linear (Jain, 1989).

Texture is intuitively viewed as Texture is intuitively viewed as descriptor in providing a measure of properties descriptor in providing a measure of properties such as smoothness, coarseness, and regularitysuch as smoothness, coarseness, and regularity (Gonzales and Woods, 1990). (Gonzales and Woods, 1990).

Texture is an attribute representing the Texture is an attribute representing the spatial arrangement of the gray levels of spatial arrangement of the gray levels of the pixelsthe pixels in a region (IEEE, 1990). in a region (IEEE, 1990).

Texture is Texture is both grey level of a single pixel and its surrounding pixelsboth grey level of a single pixel and its surrounding pixels, which was , which was coined as a unit texture, texels. These texels conformed repetitive patterns that coined as a unit texture, texels. These texels conformed repetitive patterns that dictated the effective texture analysis approach (Karu et al, 1996). dictated the effective texture analysis approach (Karu et al, 1996).

a.a. Patterns which characterize objectsPatterns which characterize objects are called texture in image processing are called texture in image processing (Jähne, 2005).(Jähne, 2005).

Technical DefinitionTechnical Definition

1.1. Texture has no single definition. Texture has no single definition.

2.2. Definitions from previous literature dedicated in Definitions from previous literature dedicated in studying texturestudying texture

3.3. The first three definitions, tells us texture is The first three definitions, tells us texture is composed of a building block that is spatially composed of a building block that is spatially arranged based on the placement rule (periodic, arranged based on the placement rule (periodic, quasi periodic, or random): like a brick a single quasi periodic, or random): like a brick a single brick is the building block, the arrangement of brick is the building block, the arrangement of the bricks that gives rise to a texture of a brick the bricks that gives rise to a texture of a brick wallwall

4.4. Texture is descriptors for smoothness, Texture is descriptors for smoothness, coarseness, and regularity coarseness, and regularity

5.5. In computer visionIn computer vision

6.6. Spatial arrangement of gray levels of the pixelSpatial arrangement of gray levels of the pixel

7.7. PatternPattern

Texture and Human Vision Texture and Human Vision SystemSystem

Pre-attentive visual system-1962-1981Pre-attentive visual system-1962-1981 Dr. Julesz Dr. Julesz

NeuroscientistNeuroscientist Texture perceptionTexture perception

Statistical approachStatistical approach Disproved conjecture that second-order is Disproved conjecture that second-order is

processed in the vision systemprocessed in the vision system TextonsTextons

ContrastContrast Terminator-end of lines, cornersTerminator-end of lines, corners Elongated blobs of different sizes - granularityElongated blobs of different sizes - granularity

1.1. As one of the hypothesis. Texture characterization emanate As one of the hypothesis. Texture characterization emanate from visual system closely emulates expertsfrom visual system closely emulates experts

2.2. Neuroscientist, studied perception of textureNeuroscientist, studied perception of texture

3.3. Before disproving, he conjectured that second-order Before disproving, he conjectured that second-order statistics is processed in the vision system, and He claimed statistics is processed in the vision system, and He claimed that two textures with similar second-order statistic is not that two textures with similar second-order statistic is not pre-attentively recognizable. pre-attentively recognizable.

4.4. In other words without close inspections, two different In other words without close inspections, two different texture with same sec stat would seem to look similar.texture with same sec stat would seem to look similar.

5.5. After series of experiments, he finally suggested that After series of experiments, he finally suggested that textons are the major player for texture discrimination. textons are the major player for texture discrimination.

6.6. And the textons are contrast, terminators. granularityAnd the textons are contrast, terminators. granularity

Texture discriminationTexture discrimination

Second-order statistics Textons

Textons instead of second-order statistics that cause the texture discrimination

The image on left is an example of two different The image on left is an example of two different textures with the same SO that is not pre-textures with the same SO that is not pre-

attentively detectable.attentively detectable.

The right image is two different textures with the The right image is two different textures with the same SO but pre-attentively detectable. same SO but pre-attentively detectable.

Among others this leads to the final statement Among others this leads to the final statement texture discrimination is made possible through the texture discrimination is made possible through the

textons.textons.

Here in this one is the difference termination of the Here in this one is the difference termination of the two texture elements .two texture elements .

In this work, the second-order statistics CoM and In this work, the second-order statistics CoM and contrast of edge elements will be explored for contrast of edge elements will be explored for

extracting visual texture properties of skin lesion. extracting visual texture properties of skin lesion.

Texture and Human Vision Texture and Human Vision SystemSystem

Frequency and OrientationFrequency and Orientation Multi-frequency and orientation analysis Multi-frequency and orientation analysis decomposition (1968) –Campbell and Robsondecomposition (1968) –Campbell and Robson Simple cells of the visual cortex respond to Simple cells of the visual cortex respond to

narrow ranges of frequency and orientation, cells narrow ranges of frequency and orientation, cells act as 2D spatial filter-(1982) De valois et al.act as 2D spatial filter-(1982) De valois et al.

Orientation-based texture segregation involves Orientation-based texture segregation involves the generation of a neural representation of the the generation of a neural representation of the surface boundary whose strength is nearly surface boundary whose strength is nearly independent of the magnitude of orientation independent of the magnitude of orientation contrast - Motoyoshi and Nishida (2001) contrast - Motoyoshi and Nishida (2001)

More studies had been conducted in part to understand human vision.More studies had been conducted in part to understand human vision.

This Campbell and Robson found that when signal received by the eye is This Campbell and Robson found that when signal received by the eye is decomposed into multiple frequencies and orientationdecomposed into multiple frequencies and orientation

Another work in the subsequent year that further support the previous Another work in the subsequent year that further support the previous finding that simple cells are highly selective/tuned to narrow frequency and finding that simple cells are highly selective/tuned to narrow frequency and orientation.orientation.

Another work found that neural representation of texture boundary is Another work found that neural representation of texture boundary is formed that is independent of magnitude and orientation of the contrastformed that is independent of magnitude and orientation of the contrast

In this work in wavelet analysis will be used for segmentation. Frequency In this work in wavelet analysis will be used for segmentation. Frequency and contrastand contrast

Texture Texture

Method DesignMethod Design

Creation of relative color images

Segmentation and morphological filtering

Relative color feature extraction

Design of tumor feature space and object feature space

Establishing statistical models from relative color features

COLORCOLOR

Create Relative Color Create Relative Color Skin Tumor ImagesSkin Tumor Images PurposePurpose

to equalize any variations caused by to equalize any variations caused by lighting, photography/printing or lighting, photography/printing or digitization processdigitization process

to equalize variations in normal skin color to equalize variations in normal skin color between individualsbetween individuals

the human visual system works on a the human visual system works on a relative color systemrelative color system

AlgorithmAlgorithm Mask out non-skin part in the image to Mask out non-skin part in the image to

calculate the normal skin colorcalculate the normal skin color Separate tumor from the imageSeparate tumor from the image Remove the skin color from the tumor to Remove the skin color from the tumor to

get a relative color skin tumor imageget a relative color skin tumor image CVIPtools functions were used to create CVIPtools functions were used to create

relative color skin tumor imagesrelative color skin tumor images

Calculate Skin ColorCalculate Skin Color

Original Noisy Skin

Tumor Image

Non-skin Algorithm

Calculate

Mask out tumor

Skin Tumor Image W/O

Noise

Average R, G, B Value

of Skin

Skin-OnlyImage

Tumor ImageTumor Image

AND

Original Noisy Skin

Tumor Image

Border Image

Tumor Image

Relative Color Tumor ImageRelative Color Tumor Image

SUBTRACT

TumorImage

Average R, G, B Value

of Skin

Relative Color Image of the Tumor

Segmentation and Segmentation and Morphological Morphological

FilteringFiltering Image segmentation was used to find regions that Image segmentation was used to find regions that represent objects or meaningful parts of objectsrepresent objects or meaningful parts of objects

Morphological filtering was used to reduce the number Morphological filtering was used to reduce the number of objects in the segmented imageof objects in the segmented image

Easy to use CVIPtools for experimenting and analysisEasy to use CVIPtools for experimenting and analysis

Feature Feature ExtractionExtraction

Relative Color Feature Relative Color Feature ExtractionExtraction

Necessary to simplify the raw image data into higher level, Necessary to simplify the raw image data into higher level, meaningful informationmeaningful information

Feature vectors are a standard technique for classifying Feature vectors are a standard technique for classifying objects, where each object is defined by a set of attributes objects, where each object is defined by a set of attributes in a feature space. in a feature space.

Totally Totally 17 color features 17 color features and binary features were and binary features were extracted using CVIPtoolsextracted using CVIPtools The three largest objects, based on the binary feature ‘area’, were The three largest objects, based on the binary feature ‘area’, were

used in feature extractionused in feature extraction Histogram featuresHistogram features, that is, color features, were extracted in each , that is, color features, were extracted in each

color band from relative color image objectscolor band from relative color image objects

17 Features17 Features

Binary featuresBinary features AreaArea

ThinnessThinness

Histogram features in R, G, B bands

Mean

Standard deviation

Skewness

Energy

Entropy

r c

crIArea ),(

24

Perimeter

AreaThinness

r c M

crIMean

),(

1

0

2 )()(L

gg gPgg

)()(1 1

0

33

gPggSkewnessL

gg

1

0

2)(L

g

gPEnergy

1

02 )(log)(

L

g

gPgPEntropy

17 Features (Cont.)17 Features (Cont.)

Design Two Feature SpacesDesign Two Feature Spaces Tumor feature spaceTumor feature space

consists of 277 feature vectors correspond to 277 skin consists of 277 feature vectors correspond to 277 skin tumor images.tumor images.

each feature vector has 51 feature elements, which are each feature vector has 51 feature elements, which are the total of 17 features of each three largest objects the total of 17 features of each three largest objects within the same tumor.within the same tumor.

Object feature spaceObject feature space had 842 feature vectors corresponding to 842 image had 842 feature vectors corresponding to 842 image

objectsobjects each feature vector has 17 feature elements, which each feature vector has 17 feature elements, which

were the binary features and color features stated as were the binary features and color features stated as aboveabove

Establishing Statistical ModelsEstablishing Statistical Models

Two feature spaces serve as two data models in order to Two feature spaces serve as two data models in order to maximize the possibility of successmaximize the possibility of success

Two classification models, Discriminant Analysis and Multi-Two classification models, Discriminant Analysis and Multi-layer Perceptron, were developed for both data modelslayer Perceptron, were developed for both data models

The training and test paradigm is used in statistical analysis The training and test paradigm is used in statistical analysis to report unbiased results of a particular algorithm to report unbiased results of a particular algorithm due to small size of data set, 282 images, we used the leave due to small size of data set, 282 images, we used the leave x x

out method, with both one and ten for out method, with both one and ten for xx

Partek software was used Partek software was used to analyze the data representing the features to analyze the data representing the features to develop a model or rules for classifying the tumorsto develop a model or rules for classifying the tumors

07/13/200507/13/2005

Computer Vision and Image Computer Vision and Image Processing Research Lab @ Processing Research Lab @

ECE Dept., SIUEECE Dept., SIUE

Quadratic Discriminant Quadratic Discriminant AnalysisAnalysis

1.1. A statistical pattern recognition technique based on A statistical pattern recognition technique based on Bayesian theory, which classifies data based on the Bayesian theory, which classifies data based on the distribution of measurement data into predefined distribution of measurement data into predefined classes classes

2.2. Normalization the feature data as preprocessingNormalization the feature data as preprocessing1.1. performed to maximize the potential of the features to performed to maximize the potential of the features to

separate classes and satisfy the requirement of the modeling separate classes and satisfy the requirement of the modeling tool such as Quadratic discriminant analysis for a Bayesian tool such as Quadratic discriminant analysis for a Bayesian distribution of the input datadistribution of the input data

3.3. Variable selection was used to choose dominant Variable selection was used to choose dominant features.features.

Multi-Layer PerceptronMulti-Layer Perceptron A feed forward neural networkA feed forward neural network

neural networks modeled after the nervous system in neural networks modeled after the nervous system in biological systems, based on the processing element the biological systems, based on the processing element the neuron neuron

widely used for pattern classification, since they learn how to widely used for pattern classification, since they learn how to transform a given data into a desired output.transform a given data into a desired output.

Principal Component Analysis (PCA) as preprocessingPrincipal Component Analysis (PCA) as preprocessing a popular multivariate technique, is to reduce dimensionality a popular multivariate technique, is to reduce dimensionality

by extracting the smallest number components that account by extracting the smallest number components that account for most of the variation in the original multivariate data and for most of the variation in the original multivariate data and to summarize the data with little loss of informationto summarize the data with little loss of information

the dispersion matrix selected for PCA in this project is the dispersion matrix selected for PCA in this project is correlationcorrelation

Multi-Layer Perceptron (Cont.)Multi-Layer Perceptron (Cont.)

Creation, training and testing of neural networksCreation, training and testing of neural networks Creation a neural network involves selection of hidden and Creation a neural network involves selection of hidden and

output neuron types and a random number generation.output neuron types and a random number generation. Four output neuron types – Softmax, Gaussian, Linear and sigmoidFour output neuron types – Softmax, Gaussian, Linear and sigmoid Three hidden neuron types – Sigmoid, Gaussian and LinearThree hidden neuron types – Sigmoid, Gaussian and Linear

Scaled Conjugate Gradient algorithm is used for learning in this Scaled Conjugate Gradient algorithm is used for learning in this project.project.

Automated and independent of user parametersAutomated and independent of user parameters Avoids time consumingAvoids time consuming Stopping criteria, sum-squared error, is selected to determine after Stopping criteria, sum-squared error, is selected to determine after

how many iterations the training should be stoppedhow many iterations the training should be stopped The trained data is then tested on itself first to examine how The trained data is then tested on itself first to examine how

far the neural network is able to classify the objects correctly.far the neural network is able to classify the objects correctly. Leave x partition out method is used for testing the algorithmLeave x partition out method is used for testing the algorithm

Experiments and Analysis in Experiments and Analysis in Object Feature SpaceObject Feature Space

Discriminant AnalysisDiscriminant Analysis 8, 9, 11 and 12 significant features were 8, 9, 11 and 12 significant features were

selected respectively for leave one out methodselected respectively for leave one out method

Number of HistogramFeatures

Area

Mean STD Skewness Energy Entropy

R G B R G B R G B R G B R G B

8 X X X X X X X X

9 X X X X X X X X X

11 X X X X X X X X X X X

12 X X X X X X X X X X X X

07/13/200507/13/2005



Experiments and Analysis in Experiments and Analysis in Tumor Feature SpaceTumor Feature Space

Discriminant AnalysisDiscriminant Analysis 24 features selected for leave ten out method24 features selected for leave ten out method

HistogramFeatures



Object 1 X X X X X X X X

Object 2 X X X X X X X

Object 3 X X X X X X X X 10 features selected for leave one out method

HistogramFeatures



Object1 X X X

Object 2 X X X X

Object 3 X X X

07/13/200507/13/2005



Experiments and Analysis in Experiments and Analysis in Tumor Feature Space (Cont.)Tumor Feature Space (Cont.)

0

10

20

30

40

50

60

70

80

90

100

DA on data with 24features usingleave 10 out

DA on data with 10features usingleave 10 out

DA on data with 24features using

leave 1 out

DA on data with 10features using

leave 1 out

Su

cces

s P

erce

nta

ge

Dys %

Mel %

Nev %

Discriminant Analysis (Cont.)Discriminant Analysis (Cont.)

07/13/200507/13/2005



Experiments and Analysis in Experiments and Analysis in Tumor Feature Space (Cont.)Tumor Feature Space (Cont.)

Multi-layer PerceptronMulti-layer Perceptron

0

10

20

30

40

50

60

70

80

90

1000 iterationsoutput layer

softmax,hidden layer

sigmod

700 iterationsouter layersoftmax,

hidden layersigmod,

700 iterationsouter layer

softmax, 17hidden layers

sigmod,


sigmod,hidden layer

sigmod

800 iterations ,output_layer


gauss

Dys correct%

Mel correct %

Nev correct %

Best features, being in the first three components of the PCA projection data, were used

Success percentages of melanoma as high as 77% and nevus is as high as 68%

Experiments and Analysis in Experiments and Analysis in Object Feature Space (Cont.)Object Feature Space (Cont.)

Discriminant Analysis (Cont.)Discriminant Analysis (Cont.)

0

10

20

30

40

50

60

70

80

90

12 Features 11 Features 9 Features 8 Features

Number of Features

Su

ce

ss

Pe

ce

nta

ge

Dys %

Mel %

Nev %

Yield consistent results in classifying melanoma from other skin tumor with above 80% success rate

07/13/200507/13/2005

Experiments and Analysis inExperiments and Analysis inObject Feature Space (Cont.)Object Feature Space (Cont.)

Multi-layer Perceptron (MLP)Multi-layer Perceptron (MLP)

0

10

20

30

40

50

60

70

80

90

100


sigmoidhidden layer

sigmod

425 iterationsouter layergaussian

hidden layergaussian

255 iterationsouter layer

linear, hiddenlayers

gaussian


softmaxhidden layer

gaussian

130 iterations ,output_layer


sigmoid

Dys correct%

Mel correct %

Nev correct %

5 out of 12 hidden-output layer neuron combinations gave better classification results

Leave one out method

Yield success percentage as high as 86% for classifying melanoma.

MLP is more consistent in classifying melanoma as well as nevus

ConclusionConclusion

Multi-Layer perceptron (MLP) with feature data Multi-Layer perceptron (MLP) with feature data preprocessed by Principal Component Analysis preprocessed by Principal Component Analysis (PCA) gave better classification results for (PCA) gave better classification results for melonoma than Discriminant Analysis (DA) melonoma than Discriminant Analysis (DA) The best overall successful rate of 78%, of which The best overall successful rate of 78%, of which

percentage correct of melanoma is 86%, nevus is percentage correct of melanoma is 86%, nevus is 62% and dysplastic is 56%. 62% and dysplastic is 56%.

The best classification results are achieved with The best classification results are achieved with sigmoid used as the hidden and output layer neuron sigmoid used as the hidden and output layer neuron

type for the MLP with PCA on Object Feature Space.type for the MLP with PCA on Object Feature Space. The three largest tumor objects are representative The three largest tumor objects are representative

for the whole skin tumor.for the whole skin tumor.

Conclusion (Cont.)Conclusion (Cont.)

However the small percentage of melanoma However the small percentage of melanoma misclassification as well as the relatively low success rate misclassification as well as the relatively low success rate for nevus and dysplastic nevi suggests that we may not for nevus and dysplastic nevi suggests that we may not have the complete data set for the experiments.have the complete data set for the experiments.

In order to achieve better classification results, future In order to achieve better classification results, future experiments experiments Needs more complete skin tumor image database.Needs more complete skin tumor image database. Should combine texture and color methods to get better Should combine texture and color methods to get better

resultsresults Will include dermoscopy imagesWill include dermoscopy images

07/13/200507/13/2005



AcknowledgementAcknowledgement

Dr. Scott E Umbaugh, SIUEDr. Scott E Umbaugh, SIUE Mr. Ragavendar SwamisaiMr. Ragavendar Swamisai Ms. Subhashini K. SrinivasanMs. Subhashini K. Srinivasan Ms. Saritha TeegalaMs. Saritha Teegala Dr. William V. Stoecker, Dr. William V. Stoecker,

Dermatologist, UMRDermatologist, UMR

07/13/200507/13/2005



Thank You!Thank You!

Yue (Iris) ChengGraduate Student

@Computer Vision and Image Processing Research

LabElectrical and Computer Engineering Department

Southern Illinois University EdwardsvilleE-mail: [email protected]

https://www.ee.siue.edu/CVIPtools

Yue (Iris) ChengGraduate Student

@Computer Vision and Image Processing Research

LabElectrical and Computer Engineering Department



CLASSIFICATION OF MALIGNANT CLASSIFICATION OF MALIGNANT MELANOMA AND DYSPLASTIC NEVI MELANOMA AND DYSPLASTIC NEVI USING IMAGE ANALYSIS: A VISUAL USING IMAGE ANALYSIS: A VISUAL

TEXTURE APPROACHTEXTURE APPROACH

Dr. Dinesh MitalDr. Dinesh Mital

University of Medicine and Dentistry of New JerseySchool of Health Related Profession

Biomedical Informatics

March 2009

Color-based Diagnosis: Color-based Diagnosis: Clinical ImagesClinical ImagesResearch Project Funded In Part by NIHResearch Project Funded In Part by NIH

Yue (Iris) Cheng, Dr. Scott E Umbaugh@

Computer Vision and Image Processing Research Lab

Electrical and Computer Engineering Department



Yue (Iris) Cheng, Dr. Scott E Umbaugh@

Computer Vision and Image Processing Research Lab

Electrical and Computer Engineering Department



9191

Spatially Constrained Segmentation of Spatially Constrained Segmentation of Dermoscopy ImagesDermoscopy Images

Howard ZhouHoward Zhou11, Mei Chen, Mei Chen22, Le Zou, Le Zou22, Richard , Richard GassGass22, ,

Laura FerrisLaura Ferris33, Laura Drogowski, Laura Drogowski33, James M. , James M. RehgRehg11

1School of Interactive Computing, Georgia Tech2Intel Research Pittsburgh

3Department of Dermatology, University of Pittsburgh

Documents

Melanoma and skin cancers vs Image Processing. 2 Skin cancer and melanoma Skin cancer : most common of all cancers Skin cancer : most common of all cancers