Embed Size (px)
Citation preview
i
Diagnosis of breast cancer
through the processing of
thermographic images and
neural networks
By
Eng. Daniela Michelle Reynoso Armenta
A dissertation submitted in partial fulfilment of
the requirements for the degree of
Master on Science with Major on Optics
at the
National Institute for Astrophysics, Optics
and Electronics
December 2017 Tonantzintla, Puebla
Thesis Advisors:
Jorge Castro Ramos PhD, INAOE
María del Pilar Gómez Gil PhD, INAOE
José Javier Báez Rojas PhD, INAOE
© INAOE 2017
The author hereby grants to INAOE permission to
reproduce and to distribute copies of this Thesis
document in whole or in part.
ii
Summary
Currently the standard method of breast cancer screening is the mastography, a
study commonly available for women of 40 years and older, because the skin and tissue
becomes thinner over time, allowing X-rays to pass through the breasts and obtain
images of any anomalies that may arise. Being an invasive and radioactive technique, it
is not comfortable for patients and therefore no regular consultations are done, thus
avoiding early diagnosis and timely treatment. In addition to the above, women outside
the age range for performing a mastography are unprotected, due to this, alternative
detection methods have been sought.
In this work we propose a method to support an early diagnosis of breast cancer through
the processing of thermographic images and the use of artificial neural networks.
Thermomastography is an innocuous test based on infrared radiation emitted by the body;
it is painless, effective, economical and fast. Our proposal is to correlate a private
database of clinical reports of patients obtained with usual and accepted techniques, such
as the mastography and biopsy, with the database of the same patients but using their
thermal imaging and thermographic classification, in this way a neuronal algorithm could
support a medical expert in the diagnosis of cases not visible without such aid. We present
the result obtained for the classification of two and three classes of cancer risks, using
the combination of various classification techniques; the neural network algorithms used
were R-CNN for Object Detection and Neural Network for Pattern Recognition. With
respect to image processing, we included Fourier Transform for pattern analysis, and
statistics based on intensity levels differences between both breasts. The final aim of this
research is eventually not just to warn if a patient could present cancer or not, but to
inform about a thermal classification indicating the possible presence of abnormalities, to
avoid future problems or to keep such patient under surveillance.
iii
Resumen
En la actualidad el método estándar de detección de cáncer de mama
es la mastografía, estudio comúnmente disponible para mujeres de 40 años
en adelante, debido a que la piel y el tejido graso se adelgaza con el paso del
tiempo, permitiendo a los rayos x atravesar las mamas y obtener imágenes de
cualquier anomalía que pudiera presentarse. Al ser una técnica invasiva y
radiactiva, no es del agrado de los pacientes y por esto no se hacen consultas
regulares, evitando así un diagnóstico temprano y un tratamiento oportuno.
Además de lo antes mencionado, las mujeres fuera del rango de edad para
realizarse una mastografía quedan desprotegidas, debido a esto se han
buscado métodos alternos de detección.
En este trabajo proponemos un método para apoyar un diagnóstico precoz de
cáncer de mama mediante el procesamiento de imágenes termográficas y el
uso de redes neuronales artificiales. La termomastografía es una prueba
inocua basada en la radiación de rayos infrarrojos que emite el cuerpo, es
indolora, eficaz, económica y rápida. Nuestra propuesta es correlacionar una
base de datos privada de informes clínicos de pacientes obtenidos con
técnicas habituales y probadas, como mastografía y biopsia, con la base de
datos de los mismos pacientes pero usando sus imágenes térmicas y
clasificación termográfica, de esta manera un algoritmo neuronal podría
encontrar patrones que por simple vista un experto médico no pudiera notar.
Presentamos el resultado obtenido para la clasificación de dos y tres clases
de riesgos de cáncer, utilizando la combinación de diferentes técnicas de
clasificación; los algoritmos de redes neuronales usados fueron R-CNN para
detección de objetos y redes neuronales de reconocimiento de patrones.
Respecto al procesamiento de imágenes, se incluyó la transformada de
Fourier para el análisis de patrones, y estadísticas basadas en las diferencias
de niveles de intensidad entre ambos senos. El objetivo final de esta
investigación es no sólo advertir si un paciente puede presentar cáncer o no,
iv
es sino informar sobre una clasificación térmica que indica la posible presencia
de anomalías, para así evitar futuros problemas o mantener a dicho paciente
bajo vigilancia.
v
Acknowledgments
To my advisers PhD. Jorge Castro Ramos, PhD. María del Pilar Gómez Gil
and PhD José Javier Báez Rojas, for their sincere interest in this project, time,
patience, and by its valuable advice. It was really nice to work with people who
put so much passion into their work and also have quality as human beings.
To my parents, for all the love and support they have always given me. I'm
sorry to make us miss each other so much because of the distance, but thanks
for believing in me and my decision to study at INAOE. Thank you for making
me as my mother says "a brave daughter".
To my family, who despite being far away always made me feel that they were
close.
To my friends from Mexicali and INAOE, for their visits and to make me feel so
dear. For supporting me for these two years of study, listening to me and giving
me courage in the most difficult times. Thank you for the shared moments,
especially your loyalty.
To my colleagues of the Biomedical Optical Instrumentation Group for their
constructive criticism regarding this work, inclusion in various projects and the
good times we spent.
To Centro de Estudios y Prevención del Cáncer (CEPREC), thank you for
entrusting us the database for the development of this thesis.
To the Consejo Nacional de Ciencia y Tecnología (CONACYT) for have given
me the master scholarship.
vi
Index
Summary ................................................................................................................. ii
Resumen ................................................................................................................ iii
Acknowledgments .................................................................................................. v
Index ....................................................................................................................... vi
1. Chapter 1: Introduction ................................................................................... 1
1.1. Background .................................................................................................... 1
1.2. Problem Statement ........................................................................................ 3
1.3. General Objective .......................................................................................... 4
1.4. Specific Objectives ......................................................................................... 5
1.5. Purpose of the Study ...................................................................................... 5
1.6. Structure ........................................................................................................ 5
2. Chapter 2: Theoretical Framework ................................................................. 7
2.1. Physical principles of thermography ............................................................... 7
2.1.1. Heat propagation: classic phenomena ........................................................... 7
2.1.1.1. Electromagnetic Spectrum ...................................................................... 8
2.1.1.2. Infrared Radiation.................................................................................... 9
2.1.1.3. Heat ...................................................................................................... 10
2.1.1.4. Temperature ......................................................................................... 10
2.1.1.5. Energy Density ...................................................................................... 11
2.1.1.6. Thermal Radiation ................................................................................. 12
2.1.1.7. Conductivity .......................................................................................... 12
2.1.1.8. Convection ............................................................................................ 13
2.1.1.9. Specific heat or heat capacity or specific heat capacity ......................... 13
2.1.1.10. Emissivity and conductivity of biological tissues .................................... 14
2.1.1.11. Heat capacity of the skin ....................................................................... 14
2.1.1.12. Fourier's Heat Equation ......................................................................... 15
2.1.2. Heat Propagation: quantic phenomena ........................................................ 16
2.1.2.1. Black Body Theory ................................................................................ 16
2.1.2.2. Planck's Law ......................................................................................... 17
2.1.2.3. Bio-heat Equation.................................................................................. 19
2.2. Technical Description of Thermography ....................................................... 21
vii
2.2.1. Thermography .............................................................................................. 21
2.2.2. IR Camera .................................................................................................... 22
2.2.2.1. Thermomastography ............................................................................. 24
2.3. Digital Image Processing .............................................................................. 27
2.3.1. Fourier Transform ........................................................................................ 28
2.3.2. Segmentation ............................................................................................... 29
2.3.3. Texture ......................................................................................................... 30
2.3.3.1. Local Binary Pattern .............................................................................. 31
2.3.4. Histogram..................................................................................................... 33
2.4. Artificial Neural Network ............................................................................... 34
2.4.1. Object Detector Using Deep Learning and R-CNN ....................................... 36
2.4.1.1. Deep Learning ...................................................................................... 36
2.4.1.2. Convolutional Neural Networks ............................................................. 37
2.4.1.3. R-CNN object detector for detecting stop signs ..................................... 39
2.4.2. Neural Network Pattern Recognition ............................................................ 40
3. Chapter 3: Methodology ................................................................................ 43
3.1. Image Acquisition ......................................................................................... 43
3.2. Pre-selection of images ................................................................................ 46
3.3. Pre-processing of images ............................................................................. 53
3.3.1. Pre-processing to detect ROI ....................................................................... 54
3.3.1.1. Image Color Code Change .................................................................... 54
3.3.1.2. Selection of Coordinates of ROI ............................................................ 55
3.3.2. Pre-Processing for Feature Extraction .......................................................... 56
3.4. R-CNN Object Detector to Automatically Detect the Right and Left Breast ... 57
3.5. Features Extraction ...................................................................................... 60
3.5.1. Texture ......................................................................................................... 61
3.5.2. Histogram..................................................................................................... 62
3.5.3. Fourier Transform ........................................................................................ 62
3.5.4. Proposed Characteristics ............................................................................. 62
3.6. Neural Network of Pattern Recognition to Classification ............................... 71
3.7. General Block Diagram of the Methodology ................................................. 73
4. Chapter 4: Results ......................................................................................... 74
4.1. ROI automatically obtained .......................................................................... 74
viii
4.1.1. ROI-testing ................................................................................................... 75
4.1.2. Frontal-123 ................................................................................................... 77
4.1.3. CorrectRanges-84 ........................................................................................ 82
4.1.4. Representatives-6 ........................................................................................ 86
4.2. ROI manually obtained and Neural Network Pattern Recognition for
classification ........................................................................................................... 88
4.3. ROI automatically obtained and Neural Network Pattern Recognition for
classification ......................................................................................................... 106
4.4. Analyses of Results .................................................................................... 111
5. Chapter 5: Conclusions ............................................................................... 112
5.1. Future Work ............................................................................................... 113
Appendix ............................................................................................................. 114
Figures ................................................................................................................ 126
Tables .................................................................................................................. 130
References .......................................................................................................... 132
1
1. Chapter 1: Introduction
This section presents an introduction to the thematic of this research, beginning with
the current context of breast cancer globally, then specifically in Mexico, and the
techniques used for its detection, afterward, the reasons that originated the realization of
this work as well as the objectives drawn up for its execution and fulfillment.
1.1. Background
Breast cancer is the most common cancer type in women in the world, including
developed and less developed countries. About 85% of breast cancers occur in women
who have no family history of breast cancer. These occur due to genetic mutations that
happen as a result of the aging process and life in general, rather than inherited mutations
(BreastCancer.org, 2017). Survival rates vary from 80% in North America, Sweden and
Japan, to around 60% in middle-income countries, and 40% in low-income countries
(Coleman, et al., 2008). The variation in these survival rates can be explained according
to the early detection programs implemented in each country, and the adequate diagnosis
and treatment facilities. Breast cancer is the second most lethal type of cancer in Mexico,
and the second leading cause of death in women aged from 24 to 30, with 14,000 new
cases each year, of which 85% are not detected in time (Gonzalez F. J., 2007); a timely
diagnosis could save 95% of those affected. Actual data suggest that around 6,000 new
cases of breast cancer occurred in Mexico in 1990 and it has increased, also there are a
lot of cases that are not detected. One of the main issues in Mexico, as elsewhere, is the
improvement and scaling up of screening to promote early detection. Only 5 to 10% of
cases in Mexico are detected in the early stages compared to 50% in the United States
(Muñoz, 2016). A study of 256 Mexican women diagnosed with breast cancer revealed
that in 90% of the cases they were the ones who identified their condition and only 10%
were diagnosed in stage I (Felicia Marie Knaul, et al., 2009).
Mammography screening is the golden standard method nowadays to diagnose breast
cancer. A mammogram is an x-ray picture of the breast and is used to detect breast
diseases like cancer. Mammograms permits to detect tumors that cannot be sensed by a
2
manual exploration. Screening mammograms can also find micro calcifications (tiny
deposits of calcium) that sometimes indicates the presence of breast cancer (NCI, 2016).
Early detection of breast cancer with screening mammography means that treatment can
be started earlier, increasing surviving rates. Results from randomized clinical trials and
other studies show that screening mammography can help reduce the number of deaths
from breast cancer among women ages 40 to 74, especially for those over age 50
(Mandelblatt, et al., 2009). When there is no signs or symptoms of a disease and a
mammogram is made, it is called screening mammogram. After a sign of symptom of the
disease has been found, it is called diagnostic mammogram. In both types of
mammograms the same machines are used, the difference resides in the time that the
patient is exposed to x-ray, diagnostic mammography has a higher radiation because
more x-ray images are needed, to obtain views of the breast from several angles. The
American College of Radiology (ACR) has established a protocol for radiologists to
describe mammogram findings. The system, called BI-RADS, has seven standardized
categories. Each BI-RADS category has a follow-up plan to manage a patient’s care
(Mandelblatt, et al., 2009), each category is shown in table 1.1.
Category Assessment Follow-up
0 Need additional imaging evaluation
Additional imaging needed before a category can be assigned
1 Negative Continue regular screening mammograms
2 Benign (noncancerous) finding
Continue regular screening mammograms
3 Probably benign Receive a 6-month follow-up mammogram
4 Suspicious abnormality May require biopsy
5 Highly suggestive of malignancy (cancer)
Requires biopsy
6 Known biopsy-proven
malignancy (cancer)
Biopsy confirms presence of cancer before treatment begins
Table 1.1 Categories of mammogram findings (Mandelblatt, et al., 2009)
The protocol to take a mammography starts with the patient undressing from the waist
up, usually standing up during the study. Each breast fits into a flat X-ray plate. The breast
is pushed down to flatten the tissue. It is recommended to hold the breath for each picture.
3
Patients mentioned that all this procedure cause discomfort and the pressure can be felt.
During the process, the doctor will revise the images as they are made.
The sensitivity (capacity to give as positive cases the really sick cases) of mammography
in the general population is believed to reside between the range 75% to 90% with a
positive predictive value of only 25% (Salhab, 2005). As a supplement information from
a mammogram and to help identifying cancers that are close to the skin, thermography
has played an important role the last few decades (Arora, 2008).
Thermography has shown to be useful, especially for women below 40 years old, which
are excluded from mammography regular studies. The Food and Drug Administration
(FDA) considers IR Thermography to be a safe practice (FDA, 2011), it is painless and
does not require the use of invasive procedures or even radiation, and there is no direct
contact with the patient.
The journal Integrative Cancer Therapies in the Breast Cancer Detection Demonstration
Project (BCDDP), showed that thermography came out as the third diagnosis technique,
after ultrasound (Kennedy, 2009). In a 2003 study conducted by Parisky et al. (Parisky,
et al., 2003), assessing the effectiveness of infrared imaging to evaluate suspicious breast
lesions, found thermography to have a 97% sensitivity (capacity to give as positive cases
the really sick cases) and positive predictive value (number of true positives divided by
the number of positive calls) of 25%. The study was a 4-year clinical trial that evaluated
875 suspicious mammographic lesions for which breast biopsy had been recommended
and mammography, with a sensitivity of only 39% and a specificity of 82%.
1.2. Problem Statement
Thermal images have been used as a complement method to diagnose breast cancer,
but only trained oncologists know how to interpret thermograms. In this moment, in most
hospitals only qualitative analysis of data are reported and accepted as a main use to
give a clinical result. It would be less subjective if the thermogram interpretation would be
based on a quantitative result, according to pattern asymmetries on temperature.
4
The infrared radiation that the human body emits has lots of information about the
metabolic process within (Nudelman, 1980). It is well known that tumors increase the
vascularity around them, making a bigger flow of blood in the area where it is located,
producing an increment in temperature. The sensors in thermographic cameras have had
an increment in thermal resolution, permitting a detection of temperature changes of 0.01
˚C. There is doubt about the deepness of heat changes a thermogram can detect,
specifically talking about screening for breast cancer, there is not enough information to
ensure that a tumor can be detected if is located profoundly or if is small (Gonzalez F. J.,
2007).
Digital Image Processing helps to increase the visualization of some important details in
thermal images. The correct treatment of a thermogram can show a clear asymmetry
pattern, and to be quantified it is necessary to traduce the image data into numerical data,
this is where the challenge appears. Many studies have tried to traduce that information
into a thermographic automatic classification, algorithms as Decision Tree, Support
Vector Machine, and others, have been used (Mookiah, Data mining technique for breast
cancer detection in thermograms using hybrid feature extraction strategy., 2012), but
giving as a result low sensitivity and specificity. Also, Artificial Neural Networks have been
implemented to classify normal and abnormal thermograms (Koay, Herry, & Frize, 2004),
but with a small number of image samples, making hard the trust in the obtained results
and leaving room for conclusions that a bigger number of images are needed to have a
better performance of the classification.
Due to its ability to extract essential information through example-based learning, Neural
Networks have been seen to be the best method for pattern recognition having a large
dataset of examples, even if the samples are not so similar between them (Lau, 1991).
1.3. General Objective
This research aims to design and to implement a model for the automatic classification of
three classes of risk of breast cancer. The model is composed of three parts: the
extraction of the interest areas (left and right breast) using a kind of neural networks based
on deep learning, the extraction of representative features and the use of Pattern
5
Recognition Neural Network to classify thermal images of breasts in two or three clinical
diagnosis.
1.4. Specific Objectives
In order to achieve the general objective, the following specific objectives were defined:
o To detect correctly the Region of Interest (ROI) of the thermograms (left and right
breast).
o To obtain the most significant asymmetric information on temperature patterns
between one breast and another, through the extraction of characteristics.
o To design a classifier based on computational intelligence techniques.
1.5. Purpose of the Study
In this thesis, a system for classification based on thermographic imaging is proposed
as an early diagnostic study of breast cancer. It is based on an innocuous test using
infrared radiation, resulting painless, effective, economical and fast. The proposals are
the combination of different classification techniques and detection algorithms, including
the feature extractions of the image to detect thermal propagation asymmetry between
left and right breast. The goal is not only to say if someone has cancer or not, but also to
know what thermal classification belongs the patient, to thereby prevent future problems
or recommend to be under surveillance. After conducting this research it was found that
the neural network is able to recognize 2 classes correctly under the conditions of
managing a correct temperature range in the thermograms, and that the medical protocol
is followed when capturing the thermographies
1.6. Structure
The thesis is organized as follows. In Chapter 1 an introduction is made to the
theme; general and specific objectives are presented. In Chapter 2 we introduce
fundamental theory of thermography principles, thermal radiation of the human body,
digital image processing, and artificial neural networks. The model proposed to classify
6
the thermograms applying digital image processing and executing neural networks is
presented in chapter 3, also the details of the methodology and algorithms used are
included. Chapter 4 presents the results obtained from the experiments carried out with
the proposed model and in chapter 6 we present the conclusions of the research, as well
as suggestions on the future work in this area.
7
2. Chapter 2: Theoretical Framework
This chapter defines some basic concepts useful to understanding this research. To
know how thermography works and what is obtained from a thermogram, it is important
to describe the physical principles of thermography and the phenomena related to them.
In this chapter, first, we present the physics behind thermography, the basic theory of
thermal energy radiation, and then, the theory of the human body thermal radiation. Also,
the technical aspects of the thermographic cameras.
To get a better visualization digital image processing was used, and it is presented in
detail in this section. The last part of chapter 2, corresponds to the explanation of artificial
neural networks, which are used in this thesis to classify the thermograms in their
respective thermic class.
2.1. Physical principles of thermography ______________________________________________________________________________
This section explains the characteristics that describe thermography, such as the
electromagnetic spectrum, infrared radiation, temperature, energy density, the concepts
of emission, conductivity and convection, as well as the emission and conductivity of
biological tissues and the heat capacity of the skin. To understand the emission of thermal
energy from the quantic point of view, the Fourier heat equation, the blackbody theory,
the Planck law and the bio-heat equation are described.
2.1.1. Heat propagation: classic phenomena
Classical physics describes phenomena whose speed is small compared to the speed
of light. In the case of the classical study of thermodynamics, it deals with energy
transformations and the thermal properties of materials (Callen, 1998). The approach is
macroscopic, it is interpreted in molecular interactions and the behavior of the medium is
considered continuous. Thanks to classical physics we can understand the concepts of
8
electromagnetic spectrum, heat, temperature and the thermal properties of biological
tissue, and the skin.
2.1.1.1. Electromagnetic Spectrum
Electromagnetic radiation is a form of periodic energy that propagates in space as time
passes, which can be heat, light, X-rays, etc., emitted by matter such as the sun, a light
bulb or the human body. An electromagnetic wave is the way in which electromagnetic
radiation propagates through space. Electromagnetic radiation is composed by two fields,
one electric (composed of electric charges at rest) and another magnetic (composed of
moving charges), which are perpendicular to each other (Hecht, 2002).
Figure 2.1 Electromagnetic wave (Radiansa_Consulting©, 2017).
Electromagnetic energy is the amount of energy stored when there is an electromagnetic
field. The energy is expressed by the intensity of the magnetic and electric field (Hecht,
2002). Electromagnetic waves have characteristics that describes them, as well as their
frequency, wavelength and amplitude. As shown in figure 2.1, the amplitude of the field
varies as the wave propagates. The frequency is the speed of oscillation of the amplitude
of the electromagnetic field, the wavelength is the distance from the maximum point of a
positive peak to the maximum point of the next positive peak, in figure 2.1 it is the distance
from point A to point B (Hecht, 2002). Both the wavelength and the frequency are
dependent characteristics, and are related through the following expression:
𝐜 = 𝛌𝐯 (1)
9
Where 𝑐=300,000,000 m/s, is the speed of electromagnetic waves in the vacuum, speed
with which electromagnetic waves propagates, λ is the wavelength and v the frequency,
which are measured in m/s, m, Hz = 1/second respectively.
The electromagnetic spectrum is the complete set of wavelengths, that is to say, it
comprises wavelengths of kilometric distances and atomic distances, as shown in figure
2.2. The range of the spectrum begins with the frequencies of 102 Hz to 106 Hz, and the
wavelength of 103m to 1m for radio waves, passing through the ultraviolet in frequencies
from 1015 Hz to 1017 Hz, and wavelengths from 380nm to 200nm, up to gamma rays with
frequencies from 1020 Hz to 1024 Hz in wavelengths from 10-2nm to 10-5nm.
Figure 2.2 Espectro electromagnético (Mini_Physics©, 2017).
2.1.1.2. Infrared Radiation
To solve the root problem in this thesis, the region of work will be the infrared region (IR)
of the electromagnetic spectrum, which is where the thermography is located. The IR
includes the region of the spectrum that goes from 3x1011 Hz to 4x1014 Hz, in wavelengths
goes from 780nm to 1mm. It is usually divided into four regions (Hecht, 2002):
Near Infrared (780-3000nm)
Intermediate Infrared (3000-6000nm)
10
Far Infrared (6000-15000nm)
Extreme Infrared (15000nm-1.0mm)
2.1.1.3. Heat
Every biological body produces heat, the distribution and measurement of this indicates
whether the body works correctly or not, to better understand this, first it is necessary to
know what heat is. The heat is the internal energy in transit (Zemansky, 1985). It is
important to note that the body does not have heat as such, if not, thermal energy, and
heat implies the transfer of it. Heat is energy that moves from a warmer system to a less
hot one, that is, as a function of a temperature difference between both. An example of
heat generation, is an applause, when the palms collide continuously thermal energy
begins to be generated, this due to the collision of particles generated by the movement
of the hands. Temperature is the measure of heat in a system. During the heat transfer
process what is known is the flow velocity of heat 𝑄, which is a function of time
(Zemansky, 1985):
𝑄 = 𝑚 ∫ 𝑐𝑑𝑡𝑡2
𝑡1 (2)
Where 𝑄 is the heat, 𝑚 the mass of the system, 𝑡 the time and 𝑐 the specific heat of the
system (amount of heat that must be contributed to the system to raise its temperature in
a temperature unit (Zemansky, 1985)); in the equation the time variable is represented as
the differential value of integration, and 𝑡1, 𝑡2 are the time laps of integration. Equation 2
indicates, that the heat only can be determined when the time t2-t1 happens.
2.1.1.4. Temperature
Thermography is a representation of the temperature of the object that is measured, in
the case of this thesis, the temperature of the breasts is measured. Temperature is a
physical property of the matter, it is a measure of the average of the kinetic energy of the
particles of a system (Weisstein, 2017).
11
From this it is deduced that at a higher average of the kinetic energy of the particles of a
system, the higher its temperature, and the lower the average of the kinetic energy, the
lower the system temperature.
The temperature is a scalar quantity, it is the measure of heat, and three units of
measurement are commonly used:
Celsius (°C)
Fahrenheit (°F)
Kelvin (°K)
In this thesis Celsius degrees are used. This scale is based on dividing into a hundred
sections the intervals between the temperature at which the water freezes and where it
evaporates. The value 0 of this scale is the freezing point of water (Zemansky, 1985). In
the Kelvin scale, the freezing point of water is at 273 degrees, and in the Fahrenheit it is
32 degrees, therefore, to make a conversion from one scale to another, a simple
subtraction is enough. The conversion formulas are shown in equation 3.
°𝐾 = °C +273°
°𝐹 = °C +32° (3)
To measure the temperature there are different types of thermometers, and depends on
the type of physical magnitude that is measured to choose the fittest. In this case, it is
needed to measure the infrared radiation, so an infrared thermometer is used (Fernández,
2017).
2.1.1.5. Energy Density
When there is more blood flow in the breasts, the temperature increased, as mentioned
above, when an object has a greater quantity of stored kinetic energy, its temperature is
higher, because it has more movement of particles. This energy stored in an object with
a certain volume is known as energy density; the electric and magnetic fields transport
and store energy (Hecht, 2002). To calculate the energy density of the electric field, the
following expression is used:
12
𝜂𝐸 =𝑒𝑛𝑒𝑟𝑔í𝑎
𝑣𝑜𝑙𝑢𝑚𝑒𝑛=
1
2𝜀0𝐸2 (4)
Where the electric permittivity in the vacuum is 𝜀0, 𝐸2 is the electric field module.
To the magnetic field the energy density is:
𝜂𝐵 =𝑒𝑛𝑒𝑟𝑔í𝑎
𝑣𝑜𝑙𝑢𝑚𝑒𝑛=
1
2
𝐵2
𝜇0 (5)
Where the magnetic permittivity in the vacuum is 𝜇0, 𝐵2 is the magnetic field module.
The total density of electromagnetic energy is the sum of the density of electrical and
magnetic energy.
𝜼 = 𝜼𝑬 + 𝜼𝑩 =𝟏
𝟐𝜺𝟎𝑬𝟐 +
𝟏
𝟐
𝑩𝟐
𝝁𝟎 (6)
In the human body, the density of electromagnetic energy gives information about the
interaction that exists with the electromagnetic fields that are formed internally, and those
of the surrounding environment.
2.1.1.6. Thermal Radiation
The phenomenon of a body emitting thermal energy due to its temperature is called
thermal radiation. The higher the temperature of a body, the greater the total energy
emitted (Zemansky, 1985). This radiated energy is electromagnetic, it can be of different
wavelengths according to its temperature (Zemansky, 1985), for example, the human
body radiates electromagnetic waves of the infrared range.
2.1.1.7. Conductivity
When a body has different temperatures at two separated points at a certain distance,
there will be a transfer of energy from the area with the highest temperature to the lowest
temperature through the matter by which the body is composed, this phenomenon is
known as conductivity (Zemansky, 1985). When a body or substance has great thermal
conductivity it is said to be a thermal conductor, otherwise it is said to be a thermal
insulator (Zemansky, 1985). In the human body, the conductivity depends on the amount
13
of water contained in each tissue, for example the muscles have a high water content,
which makes them good conductors, whereas tendons have a lower concentration of this
liquid, which categorizes them as insulators (Vega, 2017).
2.1.1.8. Convection
As explained previously, the higher the blood flow, the higher the temperature, this
phenomenon has convection as one of its physical principles. Convection is the transport
of heat by the movement of a fluid between zones with different temperatures (Zemansky,
1985). An example is a pot of water in contact with the fire of a burner, the water in the
bottom will have a higher temperature than the surface, this will cause it to rise, creating
a flow movement from the bottom of the pot towards the surface , managing to reduce
the temperature gradient between these two points. In the human body the heat transport
is given from the innermost part to the surface, by conduction and convection (Vega,
2017).
2.1.1.9. Specific heat or heat capacity or specific heat
capacity
Biological tissues have the property of changing their temperature over time. If a system
changes temperature, during a period of time, while transferring Q units of heat, it is said
to have heat capacity (Zemansky, 1985). When specifying the units of mass or internal
energy it is to speak of specific heat capacity, this is measured in 𝐽
𝑘𝑔. 𝐾, where 𝐽 is a Joule,
𝑘𝑔 kilograms, and 𝐾 Kelvin degrees (Zemansky, 1985). Each substance, or solid, in
general, each system has its characteristic heat capacity. In humans, the heat capacity
is similar to the water heat capacity, due to the high content of this substance in the body.
14
2.1.1.10. Emissivity and conductivity of biological tissues
Biological tissues have properties that allow them to exchange thermal energy with
the environment that surrounds them. Thanks to these characteristics it is possible to
obtain information through thermography. The heat produced by the organism is
transferred to the environment by conduction, convection and radiation. The highest
caloric loss (60%) is due to radiation (Vega, 2017). The hottest regions of the body are
those that have more molecular movement, for example, the most irrigated areas of
blood, which transports molecules that are in constant movement, which produces high
temperatures. Hot objects oscillate at high speed the charges of atoms, which causes
energy to be emitted in the form of electromagnetic waves (Vo-Dinh, 2003). Human skin
does not have a bright nature, it is considered an opaque body, which makes its emissivity
close to an ideal body that absorbs everything (e=1), its emissivity value is 0.993 (Vega,
2017). The emission spectrum of the skin is between the wavelengths of 2 to 20μm, this
corresponds to the infrared zone. Any object whose temperature exceeds absolute zero
(-273 ° C) will radiate infrared energy. The human body radiates approximately between
3000nm and 10000nm, that is, in the intermediate and far infrared regions (Hecht, 2002).
The biological tissues of the body, have low thermal conductivity, this due to the high
content of lipids, which function as an insulator. The conductivity of each tissue also
depends on the amount of water, the more water content the more conductors, for
example, the blood; the less water content the less conductivity, like grease (Vega, 2017).
2.1.1.11. Heat capacity of the skin
The ability to store heat is what allows the human body to maintain certain
temperatures in its tissues, which are reflected in the thermograms. The specific heat
capacity is the amount of heat needed to raise the temperature of a mass unit of a
substance by one degree (Salazar, 2003). The specific heat of the water is minimum at
35 °C. The specific heat capacity of water is higher than the ones of other liquids at room
temperature (Vega, 2017). Human tissues are mostly composed of water, which makes
their heat capacity high, that is, it takes a lot of heat to raise its temperature (Wille, 2017).
15
The human being maintains an internal temperature close to 37 °C, however, the
temperature of the body is not uniform, for example, the surface temperature (skin) is
between 29 and 34 °C. Body temperature is regulated through two processes: the
production of heat in the body (thermogenesis) and the loss of thermal energy of the body
(thermolysis) (Vega, 2017).
2.1.1.12. Fourier's Heat Equation
In thermography studies of patients what is sought is to find differences in
temperature, if these differences are very abrupt they are considered anomalous,
therefore the patient is not healthy; of being small and with gradual changes, it is
considered within the normal parameters and therefore belonging to a healthy patient
(Hobbins, 1987). The equation that allows to find differences of temperature, and to
predict the propagation of the heat in solid bodies along the course of the time, is the heat
equation of Fourier (Jiji Latif, 2009).
When there is a temperature difference between two points in a solid medium, the heat
is transmitted from the zone of higher temperature to the zone of lower temperature. The
Fourier heat equation describes how thermal conduction is distributed in a solid body as
a function of time and space (UNET, 2017).
𝑞 = −𝑘∇𝑇 (7)
Where the thermic conductivity is 𝑘, 𝑇 is the temperature, and 𝑞 the heat transfer. The
heat transfer has direction and magnitude. What the equation represents is the rate of
change in the flow of heat energy, per unit area across a surface, is proportional to the
negative temperature gradient at the surface. What this law says, is that when a
mechanism has two regions with different temperatures, internal energy is exchanged
between the areas, the area with more heat transfers energy to the area with less heat,
until reaching a thermal equilibrium, that is, the two regions approach gradually at a
common temperature (Puig, 1950).
16
2.1.2. Heat Propagation: quantic phenomena
Historically, classical physics was not enough to explain the emission of the
electromagnetic spectrum of a body according to its temperature, that is, to understand
each color that an ideal body (that absorbs everything) can emit only according to its
temperature. The scientist Planck, introduced the concept of the black body (ideal body)
to be able to explain it. Below is the theory of the black body, Planck's law and the bio-
heat equation, concepts that bring us closer to the reality in which the human body emits
electromagnetic radiation.
2.1.2.1. Black Body Theory
For any radiation emission acquisition system, such as thermography, it is necessary to
have a known model of reference. In the nineteenth century there was no way to explain
the mechanisms of emission of thermal radiation from hot bodies. To solve this situation,
Max Planck, proposed the theory of the black body in the year 1900. A black body, is an
object that absorbs all the energy that falls on it, without reflecting anything (Malacara,
2004).
A black body is characterize by the following laws:
Kirchhoff's Law
The emissivity is equal to the absorbance of a body in thermodynamic equilibrium
with its environment. The absorption spectral characteristics of a body when it is
cold are equal to the emission spectral characteristics when it is hot (Malacara,
2004).
Stefan Boltzmann's Law
All the radiation that hits the body is absorbed or emitted. The energy emitted per
unit area of the blackbody is directly proportional to the fourth power of the
temperature (Malacara, 2004).
Wien's displacement Law
There is an inverse relationship between the wavelength in which the maximum
value of the emittance is produced and its temperature (Malacara, 2004).
17
Stefan Boltzmann's law is based on the following equation:
𝐸 = 𝜎𝑇4 (8)
This equation describes the relationship that allows heat to be transmitted in the form of
radiation. All bodies above absolute zero (-273 ° C) emit radiation and radiate heat (de
Prada Pérez de Azpeitia, 2016).
The real bodies (gray body), have an emissivity value between 0 and 1, a black body has
an emissivity value of 1, so the equation for the gray bodies is the following:
𝐸 = 𝜀𝜎𝑇4 (9)
Objects with high emissivity have low reflectance, an example, the human body. The
temperature in objects of this type can be measured by means of thermal cameras (Fluke,
2009).
2.1.2.2. Planck's Law
Planck's law describes the electromagnetic energy emitted by a black body in thermal
equilibrium at a certain temperature. The expression of energy density, known as Planck's
law, is as follows:
𝑈(𝑤) =4𝜋ℎ𝑣3
𝑐3 .1
𝑒ℎ𝑣/𝑘𝑇−1 (10)
Where the energy density is U(w), w is 2πv, h is the Planck's constant, v is the frequency,
c the light speed, k the Boltzmann’s constant y T the black body's temperature (Barreto,
2008). The expression describes the amount of energy contained in a unit volume in a
frequency range between v and v+Δv (Malacara, 2004).
If it is wanted to know the energy that travels through the surface of the volume that
encloses the black body, per unit of time, you have (Barreto, 2008):
18
𝐵(𝑣) =𝑐
4𝜋𝑈(𝑣)
𝐵(𝑣, 𝑇) =2ℎ𝑣3
𝑐2.
1
𝑒ℎ𝑣/𝑘𝑇−1 (11)
Equation 5 is the expression of the radiation emission of a black body. Next, the graph of
the relation wavelength and temperature obtained with Planck's law is shown.
Figure 2.3 Planck Curve (Alice's_Astro_Info©, 2017).
What figure 2.3 shows, is that for each maximum intensity value, there is a characteristic
temperature and wavelength of the black body. At low temperatures the electromagnetic
emission is mainly of the infrared order, this again, can be seen in figure 2.3, as the value
of Kelvin grades decreases, it moves away from the visible spectrum and begins to enter
the infrared region. When the temperature increases, the electromagnetic emission is of
the visible range (Zemansky, 1985). Thus, Planck's law explains the color of iron melting,
of the stars or the sun.
According to Planck's experimental observations, he noted that these kinds of processes
are discrete, they must be integer multiples of the constant ℎ, known as the Planck's
19
constant (Cardona & al., 2008). The constant ℎ, refers to the relation that the energy of a
photon 𝐸 has, with respect to its wavelength 𝜆 and the speed of light 𝑐.
𝐸 =ℎ𝑐
𝜆 (12)
When introducing the constant, the results calculated theoretically based on quantum
physics, agreed with the results obtained experimentally. In this way, thanks to Planck, it
is now possible to understand the radiation emitted by a body due to its temperature.
All bodies whose temperature is above absolute zero emit some radiation, and depending
on the temperature at which it is found, more intense radiation is produced at a certain
wavelength. This relationship is expressed by the Wien displacement law as follows
(Zemansky, 1985):
𝜆𝑚𝑎𝑥𝑇 = 𝐵 (13)
Where the temperature in Kelvin grades is 𝑇, 𝜆 the wavelength and 𝐵 the constant whose
value is 2,898x10-3 m.K. Now, by applying this law to human skin it can be obtain the
wavelength that this tissue emits. Considering the body temperature at 37 °C (310 °K),
the above equation is apply:
𝜆𝑚𝑎𝑥 =𝐵
𝑇=
2,898x10−3m.K
310°K= 0.009348𝑚 ≈ 10𝜇𝑚 (14)
As a result it was obtained that the maximum radiation is given with a wavelength of
10μm, which corresponds to the infrared region within the electromagnetic spectrum.
2.1.2.3. Bio-heat Equation
To model the heat spread of a tumor, the bio-heat equation is needed. This describes
how heat propagates in biological tissues, is what gives the support of veracity to
20
thermography. The heat flow due to thermal conduction is expressed in the Fourier heat
equation, and by adding terms the bio-heat equation is written (Tuchin, 2007):
𝑝𝑐𝜕𝑇
𝜕𝑡= ∇(𝑘∇𝑇) + 𝑞𝑠 + 𝑞𝑝 + 𝑞𝑚 (15)
Where the light absorption is 𝑞𝑠, blood perfusion is 𝑞𝑝; perfusion is the slow and
continuous introduction of fluid to an organism, and 𝑞𝑚 is the metabolic activity (W/m3).
These are sources of thermal energy that were not considered in the Fourier equation,
but are basic elements of the thermal process of biological bodies. Now the rest terms
are going to be explained, 𝑝 corresponds to the tissue density (kg/m3), 𝑘 to the tissue
thermic conductivity (W/m˚C), 𝑐 is the specific heat (J/kg˚C), 𝑇 the local tissue
temperature (˚C), 𝑡 the time in which the heat transfer occurs (s) y ∇2 the Laplacian
operator (Tuchin, 2007).
The heat supplied by the blood perfusion 𝑄𝑝 is proportional to the perfusion radius 𝑤, that
is, the amount of blood the body pumps per second, 𝑐𝑏 the specific heat of the blood and
𝑇𝑎 the arterial temperature (Pennes, 1948).
𝑄𝑝 = 𝑤𝑝𝑏𝑐𝑏(𝑇𝑎 − 𝑇) (16)
The bio-heat equation can be applied to any biological tissue and surrounding layers, as
long as the continuity of temperature and the heat flow between the tissue and its adjacent
layers are ensured (Tuchin, 2007). The values obtained from this equation vary
significantly between normal tissue and tumors. For example, normal breast tissue has a
metabolic activity value of 450 W/m3 and of a tumor is of 29,000 W/m3 (Tuchin, 2007).
Using a theoretical modeling of the bio-heat equation, assuming spherical tumors,
differences of approximately 2˚C can be found between healthy tissue and tissue
surrounding the tumor. These results are also obtained with infrared camera technology
(Tuchin, 2007).
21
2.2. Technical Description of Thermography ______________________________________________________________________________
In order to understand the functioning of thermography in a technical way, this section
will describe how thermography works, the thermographic camera and one of its
applications, which is the basis of this thesis, thermomastography.
2.2.1. Thermography
Since the first chapter the word thermography was used because it is the technology that
was used in CEPREC to get the images of the heat propagation in the breasts, to analyze
the existence of tumors. Thermography is a technique to determine the infrared radiation
(IR) characteristics of an object under study. It converts the thermal energy radiated in
electrical signs that are amplified and traduced as images, as example of a thermal image
it is shown figure 2.4.
Thermal imaging is a process that involves conversions, here it is explain how thermal
imaging works (Tyson, 2017):
1. A special lens focuses the infrared light emitted by all of the objects in view.
2. The focused light is scanned by an infrared-detector. The detector elements create
a very detailed temperature pattern called a thermogram. It takes about one-
thirtieth of a second to make the thermogram.
3. The thermogram created is translated into electric impulses.
4. The impulses are sent to a signal-processing unit, which translates the information
from the elements into data for the display.
5. The combination of all the impulses from all of the elements creates the image.
22
Figure 2.4 Thermogram of a bat (Nudelman, 1980).
For a better visualization of isotherms (areas of equal temperature) a color code is used,
by a spectrum of colors where blue is cold and red/white is hot; middle temperatures are
green or yellowish. The tendency nowadays is toward automated computer image
processing to get a better visualization and the development of instrument design, which
is the harder challenge.
In biomedical applications, it is very useful because it indicates the blood supply around
the involved area. This technique has around of forty years of medical use (Agudelo,
2008).
There are new techniques proposed, those involve measuring the thermal patterns below
the epidermis (outer layer of cells covering the body) by using the near infrared or very
far infrared, where the skin becomes more transparent and thinner (Nudelman, 1980).
The most explored and clinically applied area of medical thermography is in the early
detection of breast cancer. This technique is also used in other anatomical regions to
analyze anomalous heating or cooling changes. Some examples of other medical
applications are diabetes, neurology, and pediatric orthopedics (Sandham, 2005).
2.2.2. IR Camera
An infrared camera is a device that detects infrared energy (heat) with no need of contact
and converts the heat detected into an electronic signal. Then the electronic signal is
processed to produce a thermogram (thermal image), which shows the temperature
23
values of the photographed objects. This device helps to monitor thermal performance or
heat-related problems in the studied object (© Flir® Systems, 2017).
The elements of the IR camera are the followings:
Objectives
Sensors
Battery
Display
Memory storage
Figure 2.5 Elements of an IR camera (© Flir® Systems, 2017)
The most important elements are the objectives and sensors. The objectives are an
optical system that incorporates several individual lenses. Those lenses define the field
of view of the camera, these system permits to get the adequate amount of IR to the
sensor. The minimum focus distance rates between 0.3m to 0.5m in commercial portable
cameras (© Flir® Systems, What is Infrared?, 2017). The sensor that detects IR is the
Focal Plane Array (FPA), which is an integrated circuit made of indium antimonide (InSb)
and indium gallium arsenide (InGaAs); this component of the camera converts the thermal
radiation into electrical signs, which are then processed to create an image (© Flir®
Systems, 2017).
24
An FPA is a two-dimensional detector array and therefore an area sensor. The image
resolution depends on pixels, which are the elements in the array of the FPA that capture
the IR; the resolution determines the capacity to distinguish the smallest details in the
thermal image (Sossa Azuela & Rodríguez Morales, 2011). The greater the number of
pixels in the FPA, the greater the spatial resolution of the thermogram.
Infrared thermography has had a technical progress and can be mainly attributed to the
advances in IR detectors. Usually thermographic systems cover a range from -20˚ to
250˚C, with a sensing capacity to detect temperature differences of 0.01 ˚C; spectral
range from 8 to 14μm and a resolution of 320x240 pixels (Meditherm, 2017).
2.2.2.1. Thermomastography
The study made in CEPREC was a thermomastography, a specific medical application of
the IR Camera. The thermomastography, consists in the study of breast’s heat distribution
detected by infrared radiation of the patient, an area of high temperature could mean a
tumor or another abnormally that involves a risk in health.
Figure 2.6 Thermogram of breast (Nudelman, 1980).
Although thermography is completely safe, non-invasive and does not expose the patient
to any type of dangerous radiation, it has not been accepted as an independent detector
system of malignancies (Sandham, 2005). It is well known that malignant tumors increase
25
the temperature of the surrounding skin, they are metabolically active and the veins which
drain these tumors are warmer than their normal temperature (Nudelman, 1980). Body
heat is produced by a continuous metabolic process consisting of oxidation of
carbohydrates and fats, which are the chief sources of energy (Boundless, 2017), the
regulation of the temperature depends on the homeostasis of the body. Homeostasis is
the tendency toward an equilibrium in biological systems, this includes the body
temperature, the role of the skin in this process is vital, its outer surface is about 20 ft2, or
1.9 m2, and it comprises 6% of the total body weight (Khurana, 2014). There is a transfer
of heat from the interior of the body, trough the fat and the dermal layers to the skin
surface, and there is transportation of heat from inside the body by means of blood flow.
The average temperature of the breast skin of women is about 37°C, for a room
temperature of 20°C (Nudelman, 1980).
Besides the knowledge of how the temperature of the body works, there are still some
doubts about thermomastography, one of those is to know which is the deepest distance
where a tumor that can be detected. According to the latest investigations, the current
state of the art in thermomastography shows that this study is capable of detecting 3cm
tumors deeper than 7cm from the skin surface, and tumors smaller than 0.5cm can be
detected close to the surface skin (Gonzalez F. J., 2007).
There is no international protocol established to acquire the thermograms of the breast,
but some medical doctors have proposed many protocols, and they all agree in some
terms. All the protocols mention that it is necessary to make the patient wait around ten
or fifteen minutes in a space with a controlled temperature, then the necessary amount
of images are taken (Borchartt, Conci, Lima, Resmini, & Sanchez, 2013). This control
helps to increase the accuracy of the study, if the patient is in a steady state per some
minutes in a space with a certain temperature, the body temperature can be in normal
state, avoiding heat changes because of external factors, as the sun or stress.
In CEPREC a standardized method for breast thermal image interpretation was adopted,
the thermal image classification methodology describes five classes from TH1 (normal)
26
to TH5 (severely abnormal) (Hobbins, 1987) (Gutierrez Delgado, 2011). The classes that
are assigned to each patient are described as follows:
TH1: aortic isothermal breasts, 1º C of thermal difference between each equidistant
point in left and right breast. Patterns of heat distribution of the breasts are very close
to identical when compared to each other.
TH2: moderate vascularization, thermal difference between each equidistant point in
left and right breast of 2º C.
TH3: vascular asymmetry, thermal difference between each equidistant point in left
and right breast around 2.5º C.
TH4: hyper vascularization, thermal difference between each equidistant point in left
and right breast greater than 2.5º C. Patterns of heat distribution of the breasts are
different compared to each other.
TH5: hyper vascularization, difference between each equidistant point in left and right
breast around 4.5º C. Patterns of heat distribution of the breasts are different
compared to each other.
Figure 2.7 Left: TH1 image. Right: TH5 image (TH-CEPREC, 2008-2009).
The criteria for the evaluation of thermograms consists on focusing in asymmetrical heat
patterns in the vertical axis. However, there are many variations to this criteria. The
absolute temperature is not as important as relative variations. The evaluation for cancer
detection can be based on one of the following criteria (Nudelman, 1980):
1. Temperature asymmetry between corresponding regions of each breast.
2. Asymmetrical venous patterns
3. Hot spots
27
4. Local areolar heat
5. Unilateral elevation of temperature.
Since all can be detected by asymmetrical pattern recognition techniques the analysis in
the computer is rather straightforward (Nudelman, 1980). This asymmetry can be
analyzed on a single picture element by picture element, comparison, or by groups of
picture elements which would cover a region of interest. The implementation of computer
diagnosis helps the oncologists to get more information about the thermogram. To get a
better visualization of the image, especially of the interest areas, it is necessary to
manipulate the image, improving the contrasts, edges, and other properties.
2.3. Digital Image Processing ______________________________________________________________________________
In this thesis, one of the biggest challenges was to acquire relevant information of the
thermograms that could indicate which thermal classification belongs each patient. To
accomplish this, some techniques of digital image processing were used in the analysis
of the thermograms. Digital image processing deals with the use of different computer
algorithms to obtain certain information of an image, to get better visualization, transform
it, etc., just a manipulation to get a desired result from the image. An image may be
defined as a two-dimensional function, f(x, y), where x and y are spatial coordinates, and
the amplitude out off at any pair of coordinates (x, y) is called the intensity or gray level
of the image at that point (Mandelblatt, et al., 2009). A digital image is composed by a
finite number of elements (pixels), and it can be a monochrome image (gray level) or a
combination of individual images, for example, RGB images consists of three individual
monochrome images the red (R), green (G), and blue (B) (Jahne, 1991).
There are many algorithms to manipulate an image and extract information from it. The
ones used in this thesis are described next:
28
2.3.1. Fourier Transform
One of the problems in the acquisition of the images, is that not all the patients follow the
indications of the medical doctors at the moment of taking the IR images, that is why
some thermograms show torsos not completely straights, but with a little inclination to the
left or right. Because of this little movements, it is not possible to compare pixel by pixel
in each breast area of a thermogram to find a significant difference in gray levels, and
suppose that it is an indicator of a health anomaly. To avoid this problem, it is necessary
to change the domain of the image, so the space location of the pixels does not give
equivocal results. A Fourier Transform (FT) is a mathematical operation that transforms
a temporary domain signal to a frequency domain (Fisher, Perkins, Walker, & Wolfart,
2003). In images, the signals correspond to gray levels or intensities of different rows or
columns in the image’s matrix.
Figure 2.8 Left: original image. Right: Fourier transform image.
For digital images Discrete Fourier Transform (DFT) is used. The DFT is the sampled
Fourier Transform and therefore does not contain all frequencies forming an image, but
only a set of samples. The image in the spatial and Fourier domain are of the same size
(Jahne, 1991).
For an image of size N×N, the two-dimensional DFT is given by:
𝑭(𝒌, 𝒍) = ∑ ∑ 𝒇(𝒊, 𝒋)𝒆−𝒊𝟐𝝅(𝒌𝒊
𝑵+
𝒍𝒋
𝑵)𝑵−𝟏
𝒋=𝟎𝑵−𝟏𝒊=𝟎 (17)
29
Equation 17 can be interpreted as the sum of the obtained value from each point in F(k,l)
result of the multiplication of the spatial image f(i,j), with the corresponding base function.
The Fourier Transform produces a complex number, with the real and imaginary part, or
with magnitude (amount of some physical quantity) and phase (position of a point in time
on a waveform cycle). Most of the geometric structure of the spatial domain image is
contained in the Fourier’s magnitude, so it is the part that is often used in image
processing (Fisher, Perkins, Walker, & Wolfart, 2003).
Some of the applications of a FT are:
1. Filtering
2. Remove additive noise from an image
3. Detect image orientation
4. Detect shade changes.
5. Geometric interpretation based on DFT periodicity
In this thesis the application is to detect shade changes, which are the representation of
the heat propagation; based on the description of a health person (TH1), the shade
changes in left and right breasts would be almost identical, this is a symmetry
characteristic.
2.3.2. Segmentation
According to the thermal classification criteria, the most significant areas are the ones
with higher temperature, because those can be indicators of the existence of a tumor. In
this work, one of the methods used was segmentation, used to separate the regions with
the highest temperature from the rest of the image. Segmentation is the operation at the
threshold between low-level image processing and the operations which analyze the
shape of objects. Basically, there are three concepts of segmentation: pixel-based
methods, which is the one used in this work, edge-based methods (detect edges), and
finally, region-based methods (analyze areas) (Jahne, 1991).
30
In the pixel-based method, segmentation only takes the gray value range which
characterizes the object of interest. The purpose is to separate the object from the
background. The gray value distributions of the object and background can be estimated
by histograms. An example is shown in figure 2.9.
Figure 2.9 Left: original image. Right: segmented image with threshold range [220-255] (TH-
CEPREC, 2008-2009).
Segmentation was used in thermograms to convert gray level images to binary ones, using the black color to indicate high temperature regions.
2.3.3. Texture
The distribution of heat in a thermogram shows a characteristic pattern for each patient,
this is a texture characteristic. The texture, in images, is based on identifying quantity and
types of regions with certain tonal properties in common, in addition to their spatial
location (Jähne, 1991). These regions are a set of pixels connected by an average tone
or the maximum and minimum tone of a certain space in the image. Tonal regions can
also be evaluated by their area and form (HAWLICK, 1979). In nature, textures are
variable, do not have neither a specific orientation, nor a regular periodicity (Jähne, 1991).
However, when comparing the same objects similar textures can be identified. An
example of the use of textures is the comparison of hands; the texture characteristics of
the hands of two different people can be compared, and the result will be that the
characteristics will be much more different than when comparing the hands of the same
individual.
As in the example of hands, happens the same with thermograms, when there is similarity
of texture in both breast, it indicates a healthy patient. One of the texture characteristic
used in this thesis is the Local Binary Pattern (LBP).
31
2.3.3.1. Local Binary Pattern
LBP algorithm consists of working with thresholds and weights (Sherdeck, 2017). The
threshold is a reference value, which is the central pixel, and the weights are the resulting
values of comparing the threshold with the neighboring pixels. The radius varies with
respect to the central pixel and in this way is compared with different neighborhoods
(Sherdeck, 2017), it is shown in figure 2.10.
Figure 2.10 Circularly symmetric neighbor sets for different radius a) R=1 b) R=1.5 c) R=2 (Eghtesad & Amirani, 2013).
The weights are obtained as figure 2.11, the threshold is compared with its neighbors, if
the threshold value (gp) is bigger than the pixel neighbor (gc) the resultant value, saved in
the S matrix is 0, if it is bigger the value will be 1. Then the S matrix is multiplied by a
matrix of the same size by the term of potency 2 per each neighbor pixel, then each result
is summed.
Figure 2.11 Local Binary Pattern process (García-Olalla, Alegre, Fernández-Robles, & García-Ordás, 2012)
32
In this thesis, the function extractLBPfeatures from MATLAB® was used, it gives by
default a resultant vector of 10 values, this was used to compare how similar the texture
from each breast was, and it can be seen in figure 2.12 that the values of LBP from each
breast shows more difference in a TH5 case than a TH1.
33
Figure 2.12. a) TH1 image with its LBP histogram b) TH5 image with its LBP histogram. Bars in blue belongs to the LBP values of the right breast, and yellow bars belongs to the left breast
LBP values (TH-CEPREC, 2008-2009).
This texture characteristic is helpful because is invariant to rotation, that is why it can
compared the left and right breast without the need of rotating one of the breasts.
2.3.4. Histogram
In the thermomastography study it is important to understand the distribution of the heat
that is represented in gray levels. A tool that gives this information is the histogram,
frequently used in the analysis of images. The histogram is a graph of the frequency with
which each level of intensity in the image is presented; the range of intensity levels goes
from 0 to 255. The histogram of an image can give information about the distribution of
the intensity levels, with this information it can be deduce if the image is homogeneous,
dark or too bright (Jahne, 1991). For example, the figure 2.13 shows the histogram of
both breasts of a TH5 image; the right breast highest frequencies are near 192 intensity
levels, and the distribution of the intensity levels is between the range of 80 to 195; in the
histogram of the left breast the highest frequencies are near 146 intensity levels, and the
34
distribution of the intensity levels is between the range of 20 to 230. This indicates that
the distribution of intensity levels in the left breast is wider than in the right breast, which
is an indicator of asymmetry in the heat propagation, and it can be seen in the original
grayscale images of each breast.
Figure 2.13 TH5 image and the histogram of each breast.
In this thesis, the histograms were used because a distribution of gray levels almost equal
in the right and left breast indicates that the thermogram corresponds to a healthy patient
(TH1), according to the thermal classification criteria.
2.4. Artificial Neural Network ______________________________________________________________________________
The main goal of this thesis, is to classify thermograms not based on qualitative
information given by the visual analyses of an oncologist, but in a quantitative way given
by the properties of the thermograms obtained by an IR camera and a computational
process. To give a thermal classification to the images, digital image processing was
35
implemented to get significant data from each image, and that resultant data was given
to an Artificial Neural Networks to accomplish the classification task. The Artificial Neural
Networks (ANN) are distributed parallel processing structures, it means structures that
can do more than one task at the same time; the concept is to simulate the human brain,
containing the neuron as the elemental unit. It is similar to a biological neuron in the way
that is capable of learning from experience (Vadivambal & Digvir, 2016). The operation
of the neuron involves the evaluation of a function from the input data and then the
calculation of a transfer function. A connection between two neurons it’s called synapses,
this permits to share information with other neurons and establish a communication
system with many of them. At the end of each synapse exist a component called weight,
which is adjustable during the network training (Huang, Kangas, & Rasco, 2007).
The interconnected nodes are organized in layers. The general organization of an ANN
is an input layer, which corresponds to the number of examples given to the ANN to train;
one or more hidden layers, and an output layer, which number of outputs corresponds to
the number of existing classes. In figure 2.14 it is shown an example of a neural network
architecture.
Classically there are distinguished two operational modes in neural systems: execution
and training. Training is the process where adjustments of free parameters of the network
are made. In general, this learning phase consists in determine a set of synaptic weights
Hidden layer (n) Input layer Output layer
Figure 2.14 Neural network architecture (Vadivambal & Digvir, 2016)
36
that allows the network to approach correctly the target (desired output). After the system
has been trained, usually, the weights and structure of the ANN stay fixed, ready to
process data; this is called execution phase.
Two or more neurons can be arranged in a layer, also, multiple neuron layers can been
used in a single ANN, this is what defines the architecture of the system. The architecture
of the network defines the way in which the units share information.
The two types of ANN used in this work were an Object Detector Using Deep Learning
and R-CNN (regions with convolutional neural networks) and Neural Network Pattern
Recognition, they are going to be explained deeper in the following section.
2.4.1. Object Detector Using Deep Learning and R-CNN
One of the specific objectives of this work was to detect the regions that correspond to
the left and right breast automatically, this is a difficult task, because each person is
different, so there are many types of breasts, who differ in size and shape. The algorithm
used to solve this problem was an Object Detector Using Deep Learning and R-CNN. It
is composed by two stages, the first part of the algorithm is Deep Learning, and the
second consists in CNN.
2.4.1.1. Deep Learning
The first part of the algorithm used is deep learning; it is a class of machine learning
algorithms that (Deng & Yu, 2014):
Uses a cascade of many layers of nonlinear processing units for feature extraction
and transformation.
Are based on the (unsupervised) learning of multiple levels of features or
representations of the data.
37
Deep learning adds the assumption that these layers of factors correspond to levels of
abstraction or composition (Bengio, Courville, & Vincent, 2013). This type of machine
learning needs a big amount of examples to train and give precise predictions as a result,
the inconvenient is that a big dataset is needed as input, the benefit is the high accuracy
that can be achieved.
2.4.1.2. Convolutional Neural Networks
The second part of the algorithm used is Convolutional Neural Networks (CNN), this
algorithm make the assumption that the inputs are images, and this constrain the
architecture in a more sensible way. The layers of a CNN have neurons arranged in 3
dimensions: width, height, depth. Every layer transforms the 3D input volume to a 3D
output volume of neuron activations. In the example of figure 2.15, the red input layer
holds the image, so its width and height would be the dimensions of the image, and the
depth would be 3 (Red, Green, Blue channels).
Figure 2.15 Example of Convolutional Neural Network arrangement of 3 dimensions (MathWorks®, Object Detection Using Deep Learning, 2017)
There are used three main types of layers to build ConvNet architectures: a)
Convolutional Layer, b) Pooling Layer, and c) Fully-Connected Layer (Britz, 2015).
a) Convolutional Layer: algebraically, convolution is the same operation as
multiplying polynomials whose coefficients are the elements of u and v. For
example, in figure 2.16, the convolution is made by multiplying each element
of the convolutional kernel with each matrix of the same size contained in the
biggest matrix, then the convolutional kernel is displaced one space, and the
38
process is repeated until it passes through all the big matrix (Scherer, Müller,
& Behnke, 2010).
Figure 2.16 Performing a convolutional operation (Apple_Inc.®, 2016)
b) Pooling layer: these layers subsample their input. The most common way to do
pooling is to apply a max operation to the result of each filter. The pool could be
over a window, for example, the following figure shows max pooling for a 2×2
window. Pooling reduces the output dimensionality but keeps the most salient
information (Britz, 2015).
Figure 2.17 Max pooling in CNN (MathWorks®, 2017)
c) Fully-Connected layer: this layer arrange all the layers; it has three types of layers,
an input layer (receives the input data), hidden layers, whose values are derived
from previous layers, and an output layer, whose values are derived from the last
hidden layer (Britz, 2015).
39
This algorithm, CNN, and Deep Learning are used as a basis for the detection of ROI
(Region of Interest).
2.4.1.3. R-CNN object detector for detecting stop signs
The basis code to detect ROI in this work was based on an example of the web page of
MathWorks ®. That example shows how an R-CNN object detector for detecting stop
signs was trained. The R-CNN was used to classify image regions. The block diagram of
the R-CNN is shown in figure 2.18.
Figure 2.18 Blocks diagram of the R-CNN
The deep learning was used to transfer learning to the R-CNN. With deep learning a big
collection of images were needed, 50,000 images of 32x32 pixels were obtained from
CIFAR-10 database (Krizhevsky & Hinton, 2009), the objective of this training was to
classify. Those images were labeled, each one with a class, there were 10 classes:
Airplane
Car
Bird
Cat
Deer
Dog
Frog
Sheep
40
Truck
Features were extracted using the CNN, those were the characteristics that the network
learns to predict to which class an image belongs.
The advantage of the use of transfer learning is that the network is pre-trained, and
already has learned, so is ready to solve a new classification task, with no need of another
large database to train, a few images can do the work. To make it work, the network
needs to make some small adjustments to the weights to support the new task
(MathWorks®, 2017). In this example, to do the sign detections, only 41 images were
required to train the network to detect stop signs. Each one of the 41 images is labeled
with the stop signs ROI coordinates.
The modification made to this code, consisted in given thermograms with labels of their
breast ROI coordinates as training images in the second part of the code, instead the stop
signs images.
2.4.2. Neural Network Pattern Recognition
The main objective of this work is to classify thermograms automatically, according
their thermal classification, which is given by the pattern of the heat propagated in the
mammal tissue. The tool used to classify the IR images, is the Neural Network Pattern
Recognition app from MATLAB®.
This type of network helps in pattern recognition problems. In this work, we want as
maximum 5 categories as outputs: TH1, TH2, TH3, TH4 and TH5. Also, as minimum, we
look for 2 output categories: normal and abnormal. This network permits to classify inputs
into a set of target categories (MathWorks®, 2017). This ANN is typically used to process
a static pattern and assigned it into one of several classes according to the result, in other
words, it can classify objects that are described by a vector of characteristics (Gurney,
1997).
The architecture of this network consists of a two-layer feedforward network; this
network’s architecture allows only forward connections, this mean there is no feedback.
As activation function it is used the sigmoid type, which is a function that is activated
41
between values from 0 to 1. The last layer of the network architecture is a softmax output
neurons (MathWorks®, 2017).
The architecture of the network can be seen in the figure 2.19.
Figure 2.19 General scheme of pattern recognition network architecture (MathWorks®, 2017)
The target data for pattern recognition networks consist of vectors 0 and 1, being 1 the
element who represent their class. As example, in this work case, to represent two
classes the vectors of the target data should be written as the next table shows:
Input Target
Healthy 01
Unhealthy 10 Table 2.1 Targets that represents two classes.
As another example, but with three classes, if there are three types of images, dog, cat
and horse, each one belongs to a class, therefore, it should be represented as follows:
Input Target
Cat 001
Dog 010
Horse 100 Table 2.2 Targets that represents three classes.
The inputs are vectors with positive numerical values, these represent the main
characteristics of the input elements. In table 2.3, continuing with the previous example,
each image is represented with its respective characteristics vector.
42
Image name Input Target
Cat [10,9,10,9,10] 001
Dog [8,8,7,6.5,8] 010
Horse [3,2,3.5,4,3] 100 Table 2.3 Example of assignment of inputs and outputs.
The trained is made with the algorithm scaled conjugate gradient backpropagation.
Backpropagation is used to get feedback of the performance of the training, it consists in
calculate derivatives of this performance with respect to the weight and bias variables
(MathWorks®, 2017). The training continues until these conditions occurs (MathWorks®,
2017):
The maximum number of repetitions is reached.
The maximum amount of time is exceeded.
Performance is minimized to the goal.
The performance gradient falls below the minimum gradient.
Validation performance has increased more than the maximum number of fails
since the last time it decreased.
This application allows to divide the input samples into three groups: training, validation
and test. The division of samples is made randomly. The network was used with vectors
of characteristics of thermograms as inputs, and it was trained as many times needed to
get the lowest error in classification.
43
3. Chapter 3: Methodology
This chapter presents a description of the proposed method for the thermal
classification of breast cancer diagnosis. This section describes the process of image
acquisition, pre-selection and pre-processing of the images, feature extraction, and
training of the neural network pattern recognition used.
3.1. Image Acquisition ______________________________________________________________________________
The thermogram database, was acquire from the year 2008 to 2009 in the Centro de
Estudios y Prevención del Cáncer (CEPREC), in Juchitán, Oaxaca. This database was
named TH-CEPREC.
The patients were photographed with an infrared camera model DL-700C with a
resolution of 320x240 pixels, and a spectral range of 8 to 14 μm.
To increase the accuracy and precision the images were acquired according to the
adopted protocol of CEPREC, based on international protocols of thermographic image
acquisition (Gutierrez Delgado, 2011):
1. The patient signs an informed consent document, so that their data can be used in
future studies (as in the case of this thesis).
2. The room temperature is controlled to 18ºC.
3. The patient sits and maintains a resting state for about 10 to 15 minutes.
4. The patient is asked to discover his torso, place his hands upwards touching the top
of the head and locate in front of the thermographic camera.
5. To take lateral photographs, the patient is asked to turn approximately 60º to the right
on her own vertical axis, after capturing the image she is asked to return to the initial
frontal position, and from there turn 60º to the left on his own vertical axis (see figure
3.1). In some cases, more photographs were taken if it was considered necessary,
the pixels were also scaled to obtain the best contrast in the temperature difference
of some regions.
44
Figure 3.1 a) Frontal Image, b) right lateral image and c) left lateral image. (TH-CEPREC, 2008-2009)
The images were save with DLI format (private format type of the camera), they were
converted to BMP format, and the resolution changed to 500x380 pixels.
Approximately 600 patients participated in the study, the complete clinical process
consisted of:
1. Health registry:
A series of questions were asked to the patient to obtain data about their healthy
habits (food, exercise, hygiene, etc.), as well as a history of diseases and previous
treatments (NIH, 2017).
2. Physical test:
The oncologist makes an examination of the body to verify the general health
(taking pressure, asking about symptoms, etc.). Afterwards, the breasts are
evaluated, both breasts are palpated to identify any signs of disease, such as
nodules, inflamed areas, or anything else that does not seem usual (NIH, 2017).
3. Image acquisition:
It consists of obtaining images of internal areas of the body through X-rays, which
is the gold standard for diagnosing breast cancer. Thermograms are also obtained,
which are images of the temperature distribution of the breasts. Two types of
images are obtained since it is necessary to use a known and accepted method, in
this case X-rays, to make a comparison with the results obtained with the
thermography.
4. Diagnostic tests:
45
Indicated for people with suspicion of having breast cancer, either by the symptoms
they experience or by the results of a screening test (touch test, mammography,
thermography). These tests are used to determine if there is breast cancer and, if
it exists, to determine if it has spread beyond the breast. Diagnostic tests (biopsies,
blood tests) are also used to collect more information about cancer (affected areas,
tumor size, etc.) in order to guide the doctor to decide the most suitable treatment
for the patient (Breastcancer.org, 2017).
The database was ordered by resident students of CEPREC over several years. All the
acquired information was captured in an Excel file, arranged alphabetically according to
the patient's name, with its respective thermal classification, image file name, digital file,
health record, date and place of examination, with observations and comments of being
considered necessary. In mid-March, a one-week stay was held to support the
organization of the database. Visual Basic for Applications (VBA) of Excel was used to
make applications that perform the organization of patient reports, this includes verifying
that they have images in DLI format, number of images with which each patient has and
if their digital reports exist.
The images and digital clinical reports of each patient were assigned to the folder whose
thermal classification corresponds: TH1, TH2, TH3, TH4 o TH5 (figure 3.2).
The thermal classification granted by the oncologist Dr. Francisco Gutiérrez of CEPREC
was based on the following thermo-biological criteria (Hobbins, 1987) (Gutierrez Delgado,
2011):
1. TH1, normal uniform non-vascular.
2. TH2, normal uniform vascular
3. TH3, equivocal
4. TH4, abnormal
5. TH5, severally abnormal
46
Figure 3.2 Classification of thermograms by thermo biological criteria: a) TH1, b) TH2, c) TH3, d) TH4, e) TH5 (TH-CEPREC, 2008-2009)
3.2. Pre-selection of images ______________________________________________________________________________
The objective of this thesis is to obtain a quantitative result of the thermogram, which
indicates to what classification a certain image belongs. For the classification, a neural
network was used; it is essential to ensure that the images are correct and the number of
47
images belonging to each class is the same, so the algorithm trains with reliable and
balanced data, and the prediction when entering new images will have a better
performance. Due to these reasons, a pre-selection of images was made, looking for
those that best followed the guidelines of the protocol used to acquire them and that
allowed a better performance of the neural network.
3.2.1. Frontal Images Selection
The thermographs should show only the torso of the patient, and it should be noted that
this follows the indications at the moment of capturing the frontal image, left and right
lateral. When reviewing the image files in DLI format of the TH-CEPREC database, it was
noted that not all people had their 3 images (front, left side and right side).
In addition, the lateral thermograms varied a lot, since some people turned more or less
than the indicated 60º. To avoid having differences in the input data, it was decided to
use only frontal images, since the pose of the persons coincides in all cases, that is, the
torso is completely facing the camera.
3.2.2. Data Balancing
In the prediction of results, neural network models seek to produce a minimum of error; if
there is a set of 100 examples of training, of which 99 examples are of the class A type
and 1 example is of the class B type, the classification prediction upon entering a new
example will have a tendency for class A. This is because the network had a greater
number of training examples of this class, the network deduces that it is 99% likely that
an unknown example is of the class A type, and 1% likely to be of the class B type. To
avoid that there is a tendency toward a certain exit response, since there are more
examples of training of a certain class, it is optimal to have balanced data. For example,
there is a set of examples with 3 classes, A, B and C, there should be N / 3 training
datasets for each class, assuming that N is 300, the distribution would be: 100 examples
of class A, 100 examples of the Class B and 100 examples of Class C.
48
For this reason, and that it is essential to have examples of cases with cancer, all cases
of TH5 were selected from the database, to take the number of cases as a reference. In
total, the number of people in each classification was:
Clasification Number of patients
diagnosed
TH5 13
TH4 38
TH3 285
TH2 4
TH1 235
Table 3.1 Number of people belonging to each classification
Having so few TH2 cases, it was decided not to work with this group, because of the issue
of imbalance. It is considered that a certain pattern of this kind cannot be described with
so little information, and this could generate confusion in the training and prediction of the
neural network. The TH5 class also has few cases, but it is essential for this investigation,
since they are cancerous cases.
The distribution is not homogeneous, and starting from the classification with less data,
we would have to use the information of only 13 people per class. One option to improve
balancing was to make 3 groups, also called classes, since according to their description,
the 5 thermographic categories can be distributed in the following way:
1. Healthy (TH1)
2. Healthy with hypervascularization (TH3)
3. Non-healthy (TH4 y TH5)
Dr. Francisco Gutiérrez, CEPREC oncologist, explained that the criteria for group
assignment are based on analyzing the thermograms in conjunction with the study of
biopsies and physical examination. Below are the observations that must be fulfilled in
the thermal images to belong to a specific group:
49
Healthy Group (TH1):
Healthy patients have breasts with similar heat distribution patterns among them, they do
not present tumors to the touch, nor are vascularizations perceived in the thermographies
(as shown in figure 3.2a).
Healthy with hypervascularization Group (TH3):
The second group is for people who have hypervascularization and may have slightly
different heat distribution patterns between breasts. These patients are recommended to
have frequent check-ups, at least once a year, to detect any early-stage abnormalities
that may develop (as shown in figure 3.2c).
Non-healthy Group (TH4 y TH5):
Finally, unhealthy patients are those whose heat distribution patterns are different
between both breasts, hypervascularization are seen in thermographies and have one or
more tumors, if benign, their classification is TH4, if malignant (check with biopsy) its
classification is TH5 (as shown in Figure 3.2d and 3.2e).
3.2.3. Selection of temperature ranges
As mentioned above, a guideline to acquire the images was followed, part of the
requirements is to control the temperature of the environment where the thermogram is
obtained, and that the patient is at rest for 15 minutes, this to regulate their body
temperature and not be affected by external factors (physical effort, sun exposure, stress,
etc.). Recalling the principle of the thermogram, it is known that each pixel of the image
represents a sampling of the temperature distribution of a physical space. The
thermographs obtained from the TH-CEPREC database show the temperature
distribution of the torso of the person, and capture the ambient temperature of the space
where the patient is immersed, as shown in figure 3.3. This tells that the thermogram must
show the room temperature as the minimum value, and the maximum value should be
the temperature recorded as the highest in the body.
50
Figure 3.3 . a) Areas of the body with the highest temperature (28.5 ºC) b) The blue zones
represent the temperature of the room, which is the minimum temperature recorded (19.8 ºC). (TH-CEPREC, 2008-2009)
The room temperature where the thermographic images were taken was under control so
it was the same every time a new image was acquired. CEPREC used a system of air
conditioner as a control device.
This measure allows to increase confidence in the data recorded by the thermogram,
capturing temperature information of 18ºC (room temperature) as minimum and a
maximum of 39ºC to 40ºC (maximum temperatures that the human body can reach).
Some images were outside the temperature ranges considered correct, so it was
analyzed the minimum and maximum temperature values recorded in 123 images,
selected for having good resolution. To obtain balanced data, the 123 images were
obtained with the following distribution: 41 healthy (TH1), 41 healthy with
hypervascularization (TH3) and 41 healthy (TH4 and TH5). This group of images was
called: Frontal-123.
The next step was to obtain the minimum and maximum temperature values presented
by each of Frontal-123's images. The minimum temperatures recorded were from 9 to 36
degrees Celsius, that is, 28 different temperature values. The maximum temperatures
recorded were 22 to 40 degrees Celsius, that is, 19 different temperature values. To
calculate the total number of images that recorded the same minimum and maximum
temperature values, tables 3.2 and 3.3 were generated.
In table 3.2, the first column represents the maximum temperature values recorded by
the 123 images. The second column shows the number of images that recorded a certain
51
maximum temperature value. From the third to the fifth column, a breakdown is made of
which group belongs the number of images that a certain maximum temperature value
represents.
In table 3.3, the first column represents the minimum temperature values recorded by the
123 images. The second column shows the number of images that recorded a certain
minimum temperature value. From the third to the fifth column, a breakdown is made of
which group belongs the number of images that a certain minimum temperature value
represents.
Maximum Temperature
# of images with the same maximum temperature value
Healthy (TH1)
Healthy with hypervascularization (TH3)
Non-healthy (TH4 y TH5)
22 1 1 0 0 23 0 0 0 0 24 1 0 1 0 25 1 1 0 0 26 4 1 2 1 27 6 1 4 1 28 6 2 3 1 29 16 5 7 4 30 14 4 5 5 31 19 7 6 6 32 12 2 5 5 33 25 9 4 12 34 11 4 2 5 35 3 2 1 0 36 2 0 1 1 37 1 1 0 0 38 0 0 0 0 39 0 0 0 0 40 1 1 0 0
Table 3.2 Total of images with same maximum temperature value and breakdown of number of images according to the class to which they belong.
52
Subsequently, a selection of minimum and maximum temperature values was made, this
according to the average value and standard deviation of the distribution of minimum and
maximum temperatures recorded in the 123 images (see figure 3.4). This measurement
is used so images with which the neural network is trained have the same temperature
range. The selection of images according to this criterion was that the minimum
temperature indicated in the thermogram was 22.5 ± 8.22ºC and the maximum 31 ±
5.62ºC.
Minimum Temperature
# of images with the same minimum temperature value
Healthy (TH1)
Healthy with hypervascularization (TH3)
Non-healthy (TH4 y TH5)
9 1 1 0 0 10 0 0 0 0 11 0 0 0 0 12 2 2 0 0 13 0 0 0 0 14 1 0 1 0 15 2 1 1 0 16 2 1 1 0 17 7 2 4 1 18 4 0 3 1 19 4 1 2 1 20 9 1 4 4 21 10 3 3 4 22 23 9 8 6 23 14 3 4 7 24 10 2 4 4 25 6 3 2 1 26 2 0 0 2 27 10 2 1 7 28 6 4 0 2 29 5 3 2 0 30 1 0 0 1 31 2 2 0 0 32 1 0 1 0 33 0 0 0 0 34 0 0 0 0 35 0 0 0 0 36 1 1 0 0
Table 3.3 Total of images with same minimum temperature value and breakdown of number of images according to the class to which they belong.
53
Figure 3.4 a) Minimum temperature distribution of each one of the 123 images b) Maximum temperature distribution of each one of the 123 images
Under this criterion, the total number of images selected was 84: 28 healthy (TH1), 28
healthy with hypervascularization (TH3) and 28 non-healthy (TH4 and TH5). This group
of images was called: CorrectRanges-84.
Another selection criterion, within this group CorrectRanges-84, was to select 6 images,
the 2 most visually representative of each of the 3 classes. This in order to evaluate that
having images that visually is very evident to know to which class they belong, and do not
confuse a trained eye, our method of classification does not either. This group of images
was called: Representatives-6.
3.3. Pre-processing of images ______________________________________________________________________________
Neural networks require a type of input data depending on their architecture. In this
work two types of networks were used, one of which input data are RGB images and the
other using feature vectors. The first is the R-CNN network, which is used to automatically
detect the left and right breast, which will be called Regions of Interest (ROI). The second
network is pattern recognition, which is used to classify. To enter the appropriate training
examples for each one, pre-processing must be done.
54
3.3.1. Pre-processing to detect ROI
91 images were selected (61 for training and 30 for testing) from the TH-CEPREC group
for this section, without the restriction of the data balance or the temperature ranges,
since for the correct performance of the algorithm it is of interest that breast can be clearly
distinguish the breasts and so be able to identify them automatically. This group of images
was called: ROI-training (61 training images) and ROI-testing (30 test images).
3.3.1.1. Image Color Code Change
The DALI Infrared Reporter software of the camera with which it worked showed 10
different color palettes, as shown in figure 3.5, each one consisted of assigning a color
code according to the temperature of the image.
Figure 3.5 Image of healthy person with mastectomy in left breast, visualized in the 10 different color palettes (TH-CEPREC, 2008-2009).
Due to visualization issues (a better contrast between hot and cold zones is perceived)
the palette 4 was chosen, which assigns a code commonly known as rainbow (Flir®,
2017), the pixels that correspond to the low temperature zones are observed in navy blue
and the pixels that correspond to the high temperature zones are observed in white, other
55
colors are assigned to the intermediate temperatures as shown in the color bar of figure
3.6
Figure 3.6 Image of patient with cancer in left breast, the color code of palette 4 is used (TH-
CEPREC, 2008-2009).
The convenience of using this color code is directly related to the way in which the R-
CNN algorithm works, which is presented later in section 3.4, whose input data is RGB
images.
3.3.1.2. Selection of Coordinates of ROI
The selection of ROI was done in two ways, a manual to train the neural network, and the
other, once the network was trained, was done automatically, the network selected the
coordinates of the ROI. The results were analyzed using both methods.
To train the network with known data, the breasts were manually delimited, and the ROI
coordinates of each image of the ROI-testing set were saved (see figure 3.7).
In an Excel® file it was indicated the name of each image and the corresponding
coordinates of its ROI (left and right breast). These coordinates, together with their
corresponding images are entered into the R-CNN to be trained, and once the network
learns it is possible to automatically detect the ROI of new images.
56
To avoid taking information that did not correspond to the breasts, it was decided to focus
only on the Regions of Interest (ROI), that is, the left and right breast. In order not to do
this selection manually at all times, it was decided to use an R-CNN network (explained
in section 3.4), so that it serves as a tool for oncologists, and when it is desired to train
the pattern recognition network with a database of hundreds of images, a faster
identification of ROI can be made.
3.3.2. Pre-Processing for Feature Extraction
For the extraction of characteristics, which is presented later in section 3.5, it is easier to
work with grayscale images, so the palette 4 was used (enhances contrasts, allows a
better distinction of hot and cold areas) for the groups Frontal-123, CorrectRanges-84
and Representative-6, and then the function rgb2gray was applied to them in MATLAB®
(example in figure 3.8). The function rgb2gray converts RGB images in gray scale by
means of a sum of the intensity value of each channel multiplying each one by a
coefficient (coefficients obtained from the calculation of luminescence); the formula is
shown next (MathWorks®, 2017):
(0.2989 ∗ 𝑅) + (0.5870 ∗ 𝐺) + (0.1140 ∗ 𝐵)ació n (18)
[X1, Y1, X2, Y2] [X’1, Y’1,
X’2, Y’2]
X2,
Y2
X’2, Y’2
X1,
Y1 X’1, Y’1
Figure 3.7 Method used to select the ROIs and obtain their coordinates (TH-CEPREC, 2008-2009).
57
Figure 3.8 Image of patient with cancer in left breast, visualized in grayscale (TH-CEPREC,
2008-2009).
In the extraction of characteristics, it is also important to focus only on the breasts, so the
breasts were manually delimited, and the coordinates of the ROI of each image of each
group of images were saved.
3.4. R-CNN Object Detector to Automatically Detect the Right and Left Breast
______________________________________________________________________________
As discussed in section 3.3.1, it was decided to use an R-CNN network, so that it
automatically detects ROI.
MathWorks® has an example program that shows how to train an object detector using
deep learning and R-CNN (Regions with Convolutional Neural Networks) using
MATLAB®, this code was modified to detect the left and right breasts of thermographic
images whose code of color is the palette 4.
The original program presented by MathWorks® shows how to train an R-CNN object
detector to detect a type of traffic signals, stop signals (MathWorks®, 2017), as is shown
in figure 3.9. This algorithm has two sections, the first one trains with a large set of images,
obtains characteristics and classifies them, the second section uses the learning acquired
by this first part, and what it does is extract features from a small set of images and detect
regions of interest.
58
Explaining in more detail, the first section of the algorithm uses a database of 50,000
RGB images of size 32x32, obtained from the CIFAR-10 database (Krizhevsky & Hinton,
2009), to classify 10 categories: airplane, automobile, bird, cat, deer, dog, frog, sheep
and truck. Convolutional filters are used to extract characteristics of each image, and thus
associate a certain pattern to each classification. The second part of the program uses
transferred learning (obtained from training with 50,000 images). Now, it only requires 41
images of stop signs and the coordinates of where those signals are for training. When
entering new images, the program is able to detect the regions where a stop signal is
found. This example is described in detail in Chapter 2, subchapter 2.4.1.
Figure 3.9 Stop signs detection by the use of R-CNN (MathWorks®, 2017).
Using the transfer learning has the advantage of reducing the number of images required
for training and training time.
The main modification to the original program consisted in using the ROI-training images
as training, instead of the 41 training images of the stop signs. To verify that the network
learned to identify the ROI correctly, the ROI-testing set was used. The objective of the
modification of the program is to detect the regions where the left and right breasts are
located, this automation allows to identify only the regions that provide vital information
to detect breast cancer or other abnormalities in the tissue.
59
The ROI-training images have assigned their corresponding manually acquired
coordinates that delimit the ROI, and it is also specified which coordinates belong to the
left and right breast.
In the MATLAB® code is used the function trainRCNNObjectDetector, to modify the
original network R-CNN, so that the neural network can classify the ROI that it detects in
each image in 3 classes: right breast, left breast and a generic background class.
The object detection method R-CNN returns the delimiting coordinates of the ROI, with a
score ranging between 0 and 1, indicating reliability in the detection, 1 indicates that the
network is 100% sure that it found the coordinates of the ROI correctly , 0 indicates that
no ROI was found in the image. Figure 3.10 shows an example of ROI detection with its
reliability value. This reliability value can be used to ignore low-scoring detections.
Figure 3.10 . a) ROI detection with reliability value of 1 b) ROI detection with reliability value of
0.832 (TH-CEPREC, 2008-2009).
60
Another modification made, to detect both breasts with less margin of error, was to
change the pixels of the first ROI detected by blacks, in this way this breast will no longer
be detected. Then the image is entered again, with its first ROI detected in black, and
thus the second ROI is detected. To make it clearer, you can see the example in figure
3.11
Figure 3.11 Left: original image. Right: image with previously detect ROI covered.
In the section of the code where breast detection is performed, the training options of the
neural network were modified to obtain a better performance. In the instruction of
MiniBatchSize it was establish the total number of training images (61), this tells the
network that it is wanted to use all the images of the training set. Also, the instruction
L2Regularization was added, this option allows to specify a learning factor in the indicated
layer, it was set to 0.0004, in other words, by reducing the learning factor, training is less
strict and over-adjustment is avoided, so better results can be obtained when testing with
images that the network does not know (MathWorks®, 2017).
In order to obtain the best ROI detection result, different tests were performed to achieve
the best combination of training image quantity, color palette of the images, training
options and number of test images. The results obtained with two different pallet colors
are shown in the appendix section B.
3.5. Features Extraction ______________________________________________________________________________
To obtain the feature vectors, which are the input data of the neural network of pattern
recognition for classification, it is necessary to extract characteristics from the training
images. These should describe the type of class to which it belongs. The approach to the
61
analysis of thermograms is the description of bilateral symmetries of the breasts, which
is a standard protocol used by medical doctors, which is the standard that will be worked
on in this thesis; a healthy person has a fairly homogeneous distribution of intensity levels
in her thermogram, and his breasts look very similar, whereas an unhealthy person has
less homogeneous intensity levels, and her breasts look different (see figure 3.12)
(Gutierrez Delgado, 2011).
As mentioned in the pre-processing of images, this section uses images in gray
scales, this to obtain information on the distribution of the intensity values of the image.
Figure 3.12 a) Thermogram of a healthy woman (TH1) b) Thermogram of a woman with cancer
on her right breast (TH5). (TH-CEPREC, 2008-2009).
For the analysis of the ROI, it was decided to be based on 3 descriptors of the image:
Texture
Histogram
Fourier Transform
3.5.1. Texture
In this thesis the texture is used to verify the similarity between breasts, since as the
theory of thermal classification says, a healthy person will have a similar temperature
distribution in both breasts (Hobbins, 1987), that in this case, is translated into a
distribution of homogeneous intensity levels. The hypothesis that this thesis work with
62
says that if both breasts have uniformly distributed intensity levels, their textures will be
similar.
3.5.2. Histogram
For breast analysis, the histogram was interpreted as a symmetry descriptor for the
distribution of gray levels, that is, if a person is healthy, his left and right breast histogram
should be similar.
3.5.3. Fourier Transform
The Fourier transform of each ROI is obtained, by transforming the domain of the
frequencies we have information of the image without being relevant the spatial location
of each pixel. This way it is obtained information from the image but invariant to the
translation (Gonzalez & Woods, 1992).
3.5.4. Proposed Characteristics
From the 3 general descriptors of the image (texture, histogram and Fourier transform),
7 types of characteristics were chosen to describe the image:
C1. Correlation of Local Binary Pattern between breasts.
A texture model is the Local Binary Pattern, several parameters of the image are
obtained, such as contrasts, corners, similar tonality zones, etc. Images of the
same objects have similar Local Binary Pattern values, so it is decided to use this
feature to see the similarity between breasts when correlating the values obtained
with the MATLAB® function extractLBPFeatures.
Feature extraction method:
A vector of 10 Local Binary Pattern (LBP) values is obtained from each breast with
the function extractLBPFeatures (V1, V2, V3, …, V10), and the absolute value of
the difference of the vectors is taken (see figure 3.13).
63
Figure 3.13 Graphical representation of how the correlation value of LBP between breasts is
obtained.
Total scalar numbers generated: 10
C2. Symmetry Value.
The image is binarized, leaving the hottest regions of both breasts in black color,
these black areas obtained in each ROI are compared to each other, under the
premise that the number of pixels in these regions should be almost same if the
person is healthy.
Feature extraction method:
A Gaussian filter is applied to each ROI (left and right breast). Gaussian filters are
masks (matrices) to attenuate white noise, by the shape of the function, the center
of each tonal region is softened leaving the edges well defined (Sossa Azuela &
Rodríguez Morales, 2012). Then, given a threshold of [220 to 255], the image is
segmented, assigning the pixels within the threshold a value of 0 and those that
are outside a 1 (a binary image is created), as an example figure 3.14 is shown.
Figure 3.14 A) Original image B) Filtered image C) Binary image (TH-CEPREC, 2008-2009).
After, each ROI is divided into 100 subdivisions (see figure 3.15), the number of
black pixels of each subdivision of the ROI is counted. Then all the values of
quantity of black pixels of each row of the matrix are summed (result matrix 1x10).
64
Finally, the amount of black pixels obtained by each ROI is compared, in figure
3.16 they are represented with the variables V.
Figure 3.15 a) Right breast ROI with 100 subdivisions c) Right breast ROI with 100 subdivisions
Figure 3.16 Graphic representation of how matrices a and b are compared.
65
Total scalar numbers generated: 10
C3. Correlation of right and left breast histograms.
It consists of acquiring the histogram of each ROI, and then obtaining its correlation
coefficient. The correlation coefficient of two variables is a measure of its linear
dependence (MathWorks®, 2017). The correlation value goes from 0 to 1, with 1
being the perfect correlation value. Healthy people tend to values close to 1 and
the sick to values close to 0.
Feature extraction method:
The histograms of the left and right breast are obtained, then the correlation value
between them is obtained (see figure 3.17).
Figure 3.17 Graphical representation of the obtaining of the correlation coefficient of the
histograms of the left and right breast (TH-CEPREC, 2008-2009).
Total scalar numbers generated: 1
C4. Difference between intensity value with greater frequency in right and left
breast.
With the histogram once acquired, the intensity value with more frequency is
searched for each ROI. In healthy people these values are the same or with a
maximum difference of 10 units of intensity levels, this value was found empirically;
in sick people the difference between these values is greater.
66
Feature extraction method:
The histograms of the left and right sine are obtained, then the intensity value with
greater frequency of each histogram is obtained, and the absolute value of the
subtraction of both is calculated.
Total scalar numbers generated: 1
C5. Intensity standard deviation in right and left breast.
The standard deviation, in a data set (gray levels of each pixel of the image) is a
measure of dispersion, which indicates how far the values move away from the
average (Mora, 2017). According to the description of each thermal class, healthy
group (TH1) people have more homogeneous ROI, so their standard deviations
must be lower than those of the non-healthy group (TH4 and TH5), and also the
standard deviation value of each ROI in healthy people should be closer.
Feature extraction method:
The standard deviation of the intensity distribution of each breast is calculated.
Total scalar numbers generated: 2
C6. Fourier transform correlation of the right and left breast.
To explain this characteristic, let's start from a basic principle. When comparing
the Fourier transforms of simple figures, like those of two squares, the result will
be two transforms with the same behavior. Now, when comparing the transforms
of two pairs, a square and rectangle, square and circle, the first pair of transforms
will be more similar than those of the second pair. Under this premise, the
transforms of the left and right breasts were compared; under the assumption that
if a healthy person is analyzed, their Fourier transforms should be very similar, that
is, having a correlation coefficient value close to 1.
67
Feature extraction method:
The Fourier transform of each ROI (left and right breasts) is obtained, the power
potencies smaller than 6.5 are filtered, there are left only the highest ones and it is
correlated the transforms of both breasts. As an example, figure 3.18 is shown.
Figure 3.18 Graphical representation of the obtaining of the correlation coefficient of the Fourier
transforms of the left and right breast (TH-CEPREC, 2008-2009).
Total scalar numbers generated: 1
C7. Correlation between quadrants I-II of each breast.
The Fourier transform has properties of symmetry, if the function of the time
domain f (x, y) is real, its Fourier transform is symmetric conjugate (Gonzalez &
Woods, 1992):
𝐹(𝑢, 𝑣) = 𝐹 ∗ (−𝑢, −𝑣)
In this way it results |𝐹(𝑢, 𝑣)| = |𝐹(−𝑢, −𝑣)|
Which indicates that the spectrum of the Fourier transform is symmetric. This
symmetry can be visualized from the origin (Bracewell, 2000). In the discrete
Fourier transform in two dimensions the quadrants I-III and II-IV are symmetrical
(Smith, 2007). Returning to the example of the basic figures, when evaluating the
Fourier transform of a square, the correlation between its quadrants I and II is 1,
since all its quadrants are equal. When evaluating a figure with a single axis of
symmetry (a triangle for example), the correlation of its quadrants I and II will be
close to 1, whereas when evaluating an anti-symmetric figure the correlation value
will decrease or even approach to 0. Landing the previous example to the thesis
work, it is started from the assumption that a healthy breast will have a
68
homogeneous gray level distribution, the shape of the breast is similar to a
parabola, which is an even function, so when correlating the Quadrants I and II of
a ROI belonging to the TH1 classification it will be obtained a value close to 1.
Feature extraction method:
The energy powers of the transform smaller than 3.2 are filtered (see figure 3.19).
Figure 3.19 a) ROI (right breast) b) Fourier Transform c) Filtered Fourier transform (TH-CEPREC, 2008-2009).
Then the ROI Fourier transform is divided in quadrants, as is shown in figure 3.20.
Figure 3.20 Quadrant division of the transform.
Finally, the correlation of quadrant I with II is made. To match, one of the quadrants
is rotated, so that its center (where the low frequencies are located), is in the same
direction as the quadrant with which it is compared (see figure 3.21).
69
Figure 3.21 Graphical representation of the obtaining of the correlation coefficient of I-II
quadrants of the Fourier transform.
Total scalar numbers generated: 2
In total, a vector with 27 positive numerical values, representing the 7 characteristics
mentioned above, is obtained from each image.
It is important to mention that some images had a visual aspect that could result in
confusion of which thermal classification it belongs, those images also obtained values
from the characteristics that represents other thermal classification. As example the table
3.4 shows the values obtained from the characteristics C3, C4 and C6, chosen because
are the easier values to compare, and the difference between thermal classes is very
notorious. There were used four images, two of classification TH1 and two TH5 that looks
as the description of their thermal classification says (rows in white color), and one image
TH1 and one TH5 that looks a little bit confusing (rows in gray color).
70
Image Name Image Thermal
Classification
C3 C4 C6
TH1_F017
TH1 0.99 0 1
TH1_F013
TH1 0.96 0 0.99
TH1_F014
TH1 0.82 18 0.54
TH4yTH5_F103
TH5 0.92 14 0.26
TH4yTH5_F112
TH5 0.79 26 0.54
TH4yTH5_F083
TH5 0.92 0 0.87
Table 3.4 Comparison of results of characteristics C3, C4 and C6 when an image is visually easy to identify to which thermal classification belongs and when it is confusing.
71
Table 3.4 shows results in the characteristic C3 that are close to 1 when the image
belongs to the TH1 classification, and lower values than .95 when it belongs to a TH5
image. The characteristic C4 values are 0 or near to that value when the image belongs
to TH1 classification, and those values are higher when the image corresponds to a
classification TH5. Finally, characteristic C6, shows values closed to 1, or even 1 when
there is an image TH1, and lower when the image belongs to TH5. According to the
description of each characteristic value, it can be seen in the table that the values of
image TH1_F014 seems to correspond to a TH5 classification, and the values of image
TH4yTH5_F083 seems to correspond to a TH1 classification. This type of results can
confuse the training of the classification, that is why it is important to obtain the most
features of an image to avoid this errors.
3.6. Neural Network of Pattern Recognition to Classification
______________________________________________________________________________
The main objective of this thesis work is to classify thermographic images
quantitatively. To complete this task, it was decided to work with the neural network
application for pattern recognition in the MATLAB® platform. This is a direct network of
two layers, with the sigmoidal function (see figure 3.22) hidden as a transfer function, and
output neurons connected by a smooth layer; it is able to classify vectors arbitrarily well,
given enough neurons in its hidden layer.
Figure 3.22 Sigmoid function.
72
In section 3.5, it is described how and what characteristics were extracted from each
image, thus obtaining vectors of characteristics that will be the inputs of the neural
network. In section 3.2, the assignment of 3 classes was explained: healthy (TH1),
healthy with hypervascularization (TH3) and non-healthy (TH4 and TH5). These classes
are the outputs of the neural network. Each image of its respective group (CorrectRanges-
84, Frontal-123 and Representatives-6) was assigned its feature vector and its output.
Once the network has the input and output data, it is proceeded to assign the percentage
of examples to train, test and validate. It is also tested with different values in the number
of hidden nodes to train the network. The nodes are the learning unit of the neural
network, they are organized in layers. The training consists of assigning weights to the
nodes of the network, so that the examples belonging to the same class are associated.
The test data, are examples that the network does not know, these allow to know the real
error committed by the neural network model. Validation helps the selection of the best
trained models (Gurney, 1997).
The neural network was trained by varying the number of nodes until it was obtained the lowest percentage of error.
73
3.7. General Block Diagram of the Methodology
______________________________________________________________________________
END
The performance
is acceptable?
Neural Network training of Pattern Recognition for Classification
Features Extraction
Pre-processing of images:
For ROI detection o Change of the color
code of the image o ROI coordinates
selection
For Features Extraction o Change of the color
code of the image o Convert the images to
grayscale
START
Thermograms acquisition
Pre-selection of images:
Frontal images selection
Data balancing
Temperature ranges selection
R-CNN to detect right and left breast
Yes
No
74
4. Chapter 4: Results
In this section are presented the results obtained from the methodology used. The first
section of this chapter shows the results of the automatic detection of the ROI (breast
area) with the algorithm R-CNN Object Detector, the second section of this chapter shows
the results of the Neural Network Pattern Recognition for Classification with ROI manually
obtained. The third section presents the results of the classification using ROI
automatically detected. The last part of this chapter presents the analyses of the results.
4.1. ROI automatically obtained ______________________________________________________________________________
To detect ROI automatically it was necessary to train the R-CNN Object Detector, as
mentioned in the methodology it was trained with the images from the group ROI-training.
To determine if the R-CNN algorithm was correctly trained and was detecting in a correct
way the breasts ROI, it was made a test with a group named ROI-testing (30 images),
after an acceptable result was achieved, it was started the detection of the ROI
automatically with each one of the images of the groups Frontal-123, CorrectRanges-84
and Representatives-6.
The results showed three cases, in this way, a visual evaluation was made giving one of
three values of score in each image of each group:
1: Breast wasn’t detected, or it was detected in a wrong area.
2: Breast was partially detected (less than 80% of the area).
3: Breast was correctly detected (at least 80% of the area).
Being 3 the desired value for the detection. Additionally, the R-CNN algorithm gives a
confidence score which represents the reliability of the detection. According with the
knowledge acquired by the network, it gives a reliability score when detecting certain
details that are supposed to determine where a breast is in the image. The network gives
scores from 0 to 1, the desire value of reliability is one. An example is shown in figure 4.1.
75
Figure 4.1 a) Image with a visual score value of 3, and a reliability score of 1 b) Image with a visual score of 2, and a reliability score of 0.999458 c) Image with a visual score of 1, and a
reliability score of 0.930862.
As mentioned in the section 3.4, the ROI detection was made in two steps, the first one
with the original image a breast is detected, then that area detected is covered in black,
it means, each pixel of the ROI detected is changed to black. Then the image with the
ROI covered is used in the algorithm to detect the other breast area.
The results of the visual evaluation and the reliability score of each image from each
group are shown in the tables in the next sections.
4.1.1. ROI-testing
The results obtained from the group ROI-testing are shown in table 4.1 and 4.2
ROI-testing
Image Visual Result
Reliability Score
1 3 1
2 3 0.999999
3 2 0.83288
4 3 1
5 3 0.999827
6 3 0.989024
7 2 0.981938
8 3 1
9 2 0.999975
10 3 1
11 3 1
12 3 0.994141
13 2 1
ROI-testing
Image Visual Result
Reliability Score
1 3 0.999992
2 1 0.999915
3 3 0.993432
4 2 0.996041
5 3 0.999999
6 2 0.999977
7 3 1
8 3 0.995978
9 2 0.999847
10 3 0.998993
11 1 0
12 2 0.999989
13 3 0.999998
76
Table 4.1 presents the results obtained from the ROI detected in the original images of
ROI-testing. The first column indicates the name of each image, at the end is presented
the average of the visual and reliability score of all the ROI detected in all the images,
and below that is the total of ROI detections who obtained 1, 2 or 3 as visual score value.
The second column contains the visual score of each ROI of each image. The third score
contains the reliability value of each ROI detected in each image.
Table 4.2 presents the results obtained from the ROI detected in the images of ROI-
testing with the ROI previously detected covered in black color. The first column indicates
the name of each image, at the end is presented the average of the visual and reliability
score of all the ROI detected in all the images, and below that is the total of ROI detections
who obtained 1, 2 or 3 as visual score value. The second column contains the visual
14 2 1
15 3 0.999765
16 3 0.999994
17 3 1
18 3 0.999999
19 3 0.999987
20 3 1
21 2 0.994879
22 3 0.999999
23 3 0.999005
24 3 0.999984
25 3 1
26 2 0.981938
27 3 0.999697
28 3 0.999999
29 3 0.995993
30 3 0.999956
Average 2.76 0.992299
Total 1: 0
Total 2: 7
Total 3: 23
14 1 0.648219
15 2 0.999555
16 3 0.999711
17 1 0.502518
18 3 0.999548
19 3 0.654088
20 3 0.999927
21 3 0.999912
22 3 0.999042
23 2 0.849207
24 2 0.999316
25 3 0.999991
26 1 0
27 1 0
28 3 0.654088
29 1 0
30 3 0.999737
Average 2.3 0.809634
Total 1: 7
Total 2: 7
Total 3: 16
Table 4.1Visual and reliability score obtained from the detection of ROI of each original image of ROI-testing.
Table 4.2 Visual and reliability score obtained from the detection of ROI of each image of ROI-testing with the ROI previously detected covered.
77
score of each ROI of each image. The third score contains the reliability value of each
ROI detected in each image.
In the first ROI detection (shown in table 4.1), the average visual score was 2.76 and the
average reliability score was 0.992299, which indicates a pretty acceptable detection from
the algorithm; 76.6% of the images got the highest value of visual score, it means, the
ROI coordinates where delimiting the breast area correctly. Also 23.3% of the images got
ROI coordinates in the correct area, but not covering all the breast region. In the second
ROI detection (shown in table 4.2), the average visual score was 2.3 and the average
reliability score was 0.809624, which indicates a lower detection in comparison with the
first detection; 53.3% of the images got the highest value of visual score. The total of
second ROI detected in the correct area, but not covering all the breast where 23.3%,
and 23.3% of the ROI were not detected.
In total, an acceptable ROI detection (visual score of 2 and 3) was made at first in 100%
of the images, and at the moment of detecting the second ROI, 76.6% of the images got
an acceptable ROI detection.
4.1.2. Frontal-123 The results obtained from each image of the group Frontal-123 are shown in table 4.3
and 4.4
Frontal-123
Image Visual Result
Reliability Score
TH1_F001 3 0.998069
TH1_F002 1 0
TH1_F003 2 0.999999
TH1_F004 1 0
TH1_F005 3 0.934467
TH1_F006 2 0.999959
TH1_F007 1 0.998203
TH1_F008 3 1
TH1_F009 3 0.967666
TH1_F010 3 0.999419
TH1_F011 3 0.999967
Frontal-123
Image Visual Result
Reliability Score
TH1_F001 3 0.999989
TH1_F002 3 0.999999
TH1_F003 3 1
TH1_F004 3 0.999882
TH1_F005 3 0.999907
TH1_F006 3 1
TH1_F007 3 0.930862
TH1_F008 3 1
TH1_F009 3 0.999961
TH1_F010 2 0.999922
TH1_F011 3 0.999996
78
TH1_F012 3 0.999156
TH1_F013 3 0.999985
TH1_F014 3 0.999760
TH1_F015 3 0.999721
TH1_F016 1 0
TH1_F017 3 0.999177
TH1_F018 1 0
TH1_F019 2 0.776823
TH1_F020 2 0.999995
TH1_F021 2 0.998548
TH1_F022 3 0.998081
TH1_F023 3 0.994459
TH1_F024 3 0.999560
TH1_F025 3 0.991986
TH1_F026 1 0
TH1_F027 3 0.998733
TH1_F028 3 0.977658
TH1_F029 1 0
TH1_F030 3 0.850580
TH1_F031 2 0.989880
TH1_F032 2 0.641392
TH1_F033 2 0.990269
TH1_F034 1 0
TH1_F035 2 0.945417
TH1_F036 2 0.855941
TH1_F037 1 0
TH1_F038 1 0
TH1_F039 3 0.991118
TH1_F040 2 0.928249
TH1_F041 1 0
TH3_F042 3 0.845211
TH3_F043 3 0.999025
TH3_F044 3 0.999401
TH3_F045 3 0.999554
TH3_F046 2 0.983678
TH3_F047 3 0.967424
TH3_F048 3 0.999790
TH3_F049 3 0.998952
TH3_F050 3 0.999962
TH3_F051 3 0.935750
TH3_F052 3 0.941525
TH3_F053 2 0.995560
TH1_F012 2 0.999852
TH1_F013 3 1
TH1_F014 2 0.999999
TH1_F015 2 0.999993
TH1_F016 3 0.999809
TH1_F017 3 0.999745
TH1_F018 1 0
TH1_F019 3 1
TH1_F020 3 1
TH1_F021 3 1
TH1_F022 3 1
TH1_F023 2 0.999999
TH1_F024 3 0.999991
TH1_F025 3 0.999994
TH1_F026 1 0
TH1_F027 3 1
TH1_F028 2 0.999944
TH1_F029 1 0
TH1_F030 3 0.999998
TH1_F031 3 1
TH1_F032 2 0.999977
TH1_F033 3 0.996914
TH1_F034 1 0
TH1_F035 2 0.999458
TH1_F036 2 1
TH1_F037 1 0
TH1_F038 3 0.999175
TH1_F039 3 1
TH1_F040 2 1
TH1_F041 1 0
TH3_F042 3 0.999895
TH3_F043 3 0.999770
TH3_F044 3 0.999998
TH3_F045 2 0.999995
TH3_F046 3 0.999999
TH3_F047 3 0.993815
TH3_F048 3 0.999989
TH3_F049 2 1
TH3_F050 3 1
TH3_F051 3 0.999996
TH3_F052 3 0.999970
TH3_F053 3 0.995560
79
TH3_F054 3 0.981829
TH3_F055 3 0.970069
TH3_F056 2 0.999994
TH3_F057 3 0.999704
TH3_F058 2 0.996295
TH3_F059 1 0
TH3_F060 3 0.998096
TH3_F061 3 0.999944
TH3_F062 2 0.780349
TH3_F063 3 0.998440
TH3_F064 3 0.999961
TH3_F065 3 0.997722
TH3_F066 2 0.906131
TH3_F067 3 0.995966
TH3_F068 3 0.999992
TH3_F069 3 0.981882
TH3_F070 3 0.991832
TH3_F071 3 0.998584
TH3_F072 2 0.999984
TH3_F073 2 0.988706
TH3_F074 2 0.975393
TH3_F075 2 0.982100
TH3_F076 3 0.999995
TH3_F077 3 0.989039
TH3_F078 3 0.977604
TH3_F079 3 0.999369
TH3_F080 2 0.966450
TH3_F081 2 0.535420
TH3_F082 2 0.999422
TH4yTH5_F083 3 0.748369
TH4yTH5_F084 3 0.998669
TH4yTH5_F085 2 0.960222
TH4yTH5_F086 3 0.999252
TH4yTH5_F087 1 0
TH4yTH5_F088 2 0.999972
TH4yTH5_F089 3 0.999999
TH4yTH5_F090 2 0.640597
TH4yTH5_F091 3 0.999962
TH4yTH5_F092 1 0
TH4yTH5_F093 3 0.993949
TH4yTH5_F094 3 0.995327
TH4yTH5_F095 3 0.999904
TH3_F054 3 0.981829
TH3_F055 3 0.970069
TH3_F056 3 0.999994
TH3_F057 2 0.999932
TH3_F058 3 1
TH3_F059 3 0.999990
TH3_F060 2 1
TH3_F061 3 1
TH3_F062 1 0.997373
TH3_F063 3 1
TH3_F064 2 1
TH3_F065 3 0.999972
TH3_F066 3 0.999665
TH3_F067 3 0.999986
TH3_F068 3 1
TH3_F069 3 1
TH3_F070 3 1
TH3_F071 3 1
TH3_F072 3 1
TH3_F073 2 0.999988
TH3_F074 2 0.999792
TH3_F075 3 0.999939
TH3_F076 3 1
TH3_F077 3 1
TH3_F078 3 0.999999
TH3_F079 3 0.999999
TH3_F080 3 0.993041
TH3_F081 3 1
TH3_F082 3 1
TH4yTH5_F083 2 0.999995
TH4yTH5_F084 3 0.999998
TH4yTH5_F085 3 0.999988
TH4yTH5_F086 3 0.999999
TH4yTH5_F087 3 0.989637
TH4yTH5_F088 3 1
TH4yTH5_F089 3 1
TH4yTH5_F090 2 0.999949
TH4yTH5_F091 3 1
TH4yTH5_F092 3 0.999853
TH4yTH5_F093 3 0.999704
TH4yTH5_F094 3 0.999999
TH4yTH5_F095 3 1
80
Table 4.4 Visual and reliability score obtained from the detection of ROI of each image of Frontal-123 with the ROI previously detected covered.
Table 4.3 presents the results obtained from the ROI detected in the original images of
Frontal-123. The first column indicates the name of each image, at the end is presented
the average of the visual and reliability score of all the ROI detected in all the images,
TH4yTH5_F096 3 0.999740
TH4yTH5_F097 3 0.822266
TH4yTH5_F098 3 0.992586
TH4yTH5_F099 1 0
TH4yTH5_F100 2 0.999755
TH4yTH5_F101 3 0.999977
TH4yTH5_F102 3 0.999867
TH4yTH5_F103 3 0.999821
TH4yTH5_F104 1 0
TH4yTH5_F105 3 0.999679
TH4yTH5_F106 2 0.998621
TH4yTH5_F107 2 0.999997
TH4yTH5_F108 2 0.999956
TH4yTH5_F109 1 0
TH4yTH5_F110 1 0
TH4yTH5_F111 1 0
TH4yTH5_F112 3 0.999103
TH4yTH5_F113 1 0
TH4yTH5_F114 1 0
TH4yTH5_F115 3 0.999984
TH4yTH5_F116 3 1
TH4yTH5_F117 1 0
TH4yTH5_F118 3 0.994519
TH4yTH5_F119 3 0.999300
TH4yTH5_F120 3 0.999988
TH4yTH5_F121 2 0.999913
TH4yTH5_F122 2 0.999990
TH4yTH5_F123 1 0
Average 2.357 0.793834
Total 1: 23
Total 2: 33
Total 3: 67
TH4yTH5_F096 3 0.999998
TH4yTH5_F097 2 0.998323
TH4yTH5_F098 3 0.999966
TH4yTH5_F099 3 0.999975
TH4yTH5_F100 3 0.999999
TH4yTH5_F101 3 1
TH4yTH5_F102 3 1
TH4yTH5_F103 3 1
TH4yTH5_F104 3 1
TH4yTH5_F105 3 0.999990
TH4yTH5_F106 3 0.999874
TH4yTH5_F107 3 1
TH4yTH5_F108 3 1
TH4yTH5_F109 3 0.999751
TH4yTH5_F110 3 1
TH4yTH5_F111 3 0.999999
TH4yTH5_F112 3 1
TH4yTH5_F113 3 0.999999
TH4yTH5_F114 3 0.999999
TH4yTH5_F115 3 0.999918
TH4yTH5_F116 3 1
TH4yTH5_F117 3 0.999938
TH4yTH5_F118 3 0.999981
TH4yTH5_F119 3 1
TH4yTH5_F120 3 1
TH4yTH5_F121 3 1
TH4yTH5_F122 3 0.999999
TH4yTH5_F123 3 0.999995
Average 2.723 0.949941
Total 1: 7
Total 2: 20
Total 3: 96
Table 4.3 Visual and reliability score obtained from the detection of ROI of each original image of Frontal-123.
81
and below that is the total of ROI detections who obtained 1, 2 or 3 as visual score value.
The second column contains the visual score of each ROI of each image. The third score
contains the reliability value of each ROI detected in each image.
Table 4.4 presents the results obtained from the ROI detected in the images of Frontal-
123 with the ROI previously detected covered in black color. The first column indicates
the name of each image, at the end is presented the average of the visual and reliability
score of all the ROI detected in all the images, and below that is the total of ROI detections
who obtained 1, 2 or 3 as visual score value. The second column contains the visual
score of each ROI of each image. The third score contains the reliability value of each
ROI detected in each image.
In figure 4.2 are shown three examples, each one with a different thermal classification
and all obtained a visual score of 3 in both ROI.
Figure 4.2 a) Image TH1_F001, with visual score of 3 in both ROI. b) Image TH3_F048, with visual score of 3 in both ROI. c) Image TH4yTH5_F112, with visual score of 3 in both ROI.
In the first ROI detection (shown in table 4.3), the average visual score was 2.72 and the
average reliability score was 0.949941, which indicates a pretty acceptable detection from
the algorithm; 78% of the images got the highest value of visual score, it means, the ROI
coordinates where delimiting the breast area correctly. Also 16.2% of the images got ROI
coordinates in the correct area, but not covering all the breast region. The total of images
which not detect a ROI were the 5.7%.In the second ROI detection (shown in table 4.4),
the average visual score was 2.35 and the average reliability score was 0.793834, which
indicates a lower detection in comparison with the first detection, but still good enough;
54.4% of the images got the highest value of visual score. The total of second ROI
detected in the correct area, but not covering all the breast where 26.8%, and 18.7% of
the ROI were not detected.
82
In total, an acceptable ROI detection (visual score of 2 and 3) was made at first in 94.2%
of the images, and at the moment of detecting the second ROI, 81.2% of the images got
an acceptable ROI detection.
4.1.3. CorrectRanges-84
The results obtained from each image of the group CorrectRanges-84 are shown in
table 4.5 and 4.6
CorrectRanges-84
Image Visual Result
Reliability Score
TH1_F001 3 0.999989
TH1_F002 3 0.999999
TH1_F006 3 1
TH1_F007 3 0.930862
TH1_F008 3 1
TH1_F009 3 0.999961
TH1_F011 3 0.999996
TH1_F012 2 0.999852
TH1_F013 3 1
TH1_F014 2 0.999999
TH1_F017 2 0.999745
TH1_F019 3 1
TH1_F020 3 1
TH1_F021 3 1
TH1_F022 3 1
TH1_F023 2 0.999999
TH1_F025 3 0.999994
TH1_F027 3 1
TH1_F029 1 0
TH1_F031 3 1
TH1_F032 2 0.999977
TH1_F033 3 0.996914
TH1_F035 2 0.999458
TH1_F037 1 0
TH1_F038 3 0.999175
TH1_F039 3 1
CorrectRanges-84
Image Visual Result
Reliability Score
TH1_F001 3 0.998069
TH1_F002 1 0
TH1_F006 2 0.999959
TH1_F007 1 0.998203
TH1_F008 3 1
TH1_F009 3 0.967666
TH1_F011 3 0.999967
TH1_F012 3 0.999156
TH1_F013 3 0.999985
TH1_F014 3 0.999760
TH1_F017 3 0.999177
TH1_F019 2 0.776823
TH1_F020 2 0.999995
TH1_F021 2 0.998548
TH1_F022 3 0.998081
TH1_F023 3 0.994459
TH1_F025 3 0.991986
TH1_F027 3 0.998733
TH1_F029 1 0
TH1_F031 2 0.989880
TH1_F032 2 0.641392
TH1_F033 2 0.990269
TH1_F035 2 0.945417
TH1_F037 1 0
TH1_F038 1 0
TH1_F039 3 0.991118
83
TH1_F040 2 1
TH1_F041 1 0
TH3_F043 3 0.999770
TH3_F044 3 0.999998
TH3_F045 2 0.999995
TH3_F047 3 0.993815
TH3_F048 3 0.999989
TH3_F049 2 1
TH3_F050 3 1
TH3_F051 3 0.999996
TH3_F052 3 0.999970
TH3_F058 3 1
TH3_F060 2 1
TH3_F061 3 1
TH3_F062 1 0.997373
TH3_F063 3 1
TH3_F064 2 1
TH3_F065 3 0.999972
TH3_F066 3 0.999665
TH3_F067 3 0.999986
TH3_F068 3 1
TH3_F070 3 1
TH3_F071 3 1
TH3_F072 3 1
TH3_F075 3 0.999939
TH3_F076 3 1
TH3_F077 3 1
TH3_F079 3 0.999999
TH3_F081 3 1
TH3_F082 3 1
TH4yTH5_F083 2 0.999995
TH4yTH5_F085 3 0.999988
TH4yTH5_F086 3 0.999999
TH4yTH5_F088 3 1
TH4yTH5_F089 3 1
TH4yTH5_F092 3 0.999853
TH4yTH5_F093 3 0.999704
TH4yTH5_F095 3 1
TH4yTH5_F096 3 0.999998
TH4yTH5_F097 2 0.998323
TH4yTH5_F098 3 0.999966
TH4yTH5_F101 3 1
TH1_F040 2 0.928249
TH1_F041 1 0
TH3_F043 3 0.999025
TH3_F044 3 0.999401
TH3_F045 3 0.999554
TH3_F047 3 0.967424
TH3_F048 3 0.999790
TH3_F049 3 0.998952
TH3_F050 3 0.999962
TH3_F051 3 0.935750
TH3_F052 3 0.941525
TH3_F058 2 0.996295
TH3_F060 3 0.998096
TH3_F061 3 0.999944
TH3_F062 2 0.780349
TH3_F063 3 0.998440
TH3_F064 3 0.999961
TH3_F065 3 0.997722
TH3_F066 2 0.906131
TH3_F067 3 0.995966
TH3_F068 3 0.999992
TH3_F070 3 0.991832
TH3_F071 3 0.998584
TH3_F072 2 0.999984
TH3_F075 2 0.982100
TH3_F076 3 0.999995
TH3_F077 3 0.989039
TH3_F079 3 0.999369
TH3_F081 2 0.535420
TH3_F082 2 0.999422
TH4yTH5_F083 3 0.748369
TH4yTH5_F085 2 0.960222
TH4yTH5_F086 3 0.999252
TH4yTH5_F088 2 0.999972
TH4yTH5_F089 3 0.999999
TH4yTH5_F092 1 0
TH4yTH5_F093 3 0.993949
TH4yTH5_F095 3 0.999904
TH4yTH5_F096 3 0.999740
TH4yTH5_F097 3 0.822266
TH4yTH5_F098 3 0.992586
TH4yTH5_F101 3 0.999977
84
Table 4.6 Visual and reliability score obtained from the detection of ROI of each image of CorrectRanges-84 with the ROI previously detected covered.
Table 4.5 presents the results obtained from the ROI detected in the original images of
CorrectRanges-84. The first column indicates the name of each image, at the end is
presented the average of the visual and reliability score of all the ROI detected in all the
images, and below that is the total of ROI detections who obtained 1, 2 or 3 as visual
score value. The second column contains the visual score of each ROI of each image.
The third score contains the reliability value of each ROI detected in each image.
Table 4.6 presents the results obtained from the ROI detected in the images of
CorrectRanges-84 with the ROI previously detected covered in black color. The first
column indicates the name of each image, at the end is presented the average of the
visual and reliability score of all the ROI detected in all the images, and below that is the
total of ROI detections who obtained 1, 2 or 3 as visual score value. The second column
contains the visual score of each ROI of each image. The third score contains the
reliability value of each ROI detected in each image.
TH4yTH5_F102 3 1
TH4yTH5_F103 3 1
TH4yTH5_F105 3 0.999990
TH4yTH5_F109 3 0.999751
TH4yTH5_F110 3 1
TH4yTH5_F111 3 0.999999
TH4yTH5_F112 3 1
TH4yTH5_F113 3 0.999999
TH4yTH5_F114 3 0.999999
TH4yTH5_F115 3 0.999918
TH4yTH5_F116 3 1
TH4yTH5_F117 3 0.999938
TH4yTH5_F120 3 1
TH4yTH5_F121 3 1
TH4yTH5_F122 3 0.999999
TH4yTH5_F123 3 0.999995
Average 2.75 0.963259
Total 1: 4
Total 2: 13
Total 3: 67
TH4yTH5_F102 3 0.999867
TH4yTH5_F103 3 0.999821
TH4yTH5_F105 3 0.999679
TH4yTH5_F109 1 0
TH4yTH5_F110 1 0
TH4yTH5_F111 1 0
TH4yTH5_F112 3 0.999103
TH4yTH5_F113 1 0
TH4yTH5_F114 1 0
TH4yTH5_F115 3 0.999984
TH4yTH5_F116 3 1
TH4yTH5_F117 1 0
TH4yTH5_F120 3 0.999988
TH4yTH5_F121 2 0.999913
TH4yTH5_F122 2 0.999990
TH4yTH5_F123 1 0
Average 2.42 0.818232
Total 1: 14
Total 2: 20
Total 3: 50
Table 4.5 Visual and reliability score obtained from the detection of ROI of each original image of CorrectRanges-84.
85
In figure 4.3 are shown three examples, each one with a different thermal classification
and all obtained a visual score of 3 in both ROI.
Figure 4.3 a) Image TH1_F009, with visual score of 3 in both ROI. b) Image TH3_F044, with visual score of 3 in both ROI. c) Image TH4yTH5_F120, with visual score of 3 in both ROI.
In the first ROI detection (shown in table 4.5), the average visual score was 2.75 and the
average reliability score was 0.963259, which is a better average than the one obtained
with the group Frontal-123, it could indicates that the selection of temperature ranges
affects the detection of ROI; 79.76% of the images got the highest value of visual score,
it means, the ROI coordinates where delimiting the breast area correctly. Also 15.4% of
the images got ROI coordinates in the correct area, but not covering all the breast region.
The total of images which not detect a ROI were the 4.7%.In the second ROI detection
(shown in table 4.6), the average visual score was 2.42 and the average reliability score
was 0.818232, which indicates a lower detection in comparison with the first detection,
but higher than the average scores of the second detection of the group Frontal-123;
59.5% of the images got the highest value of visual score. The total of second ROI
detected in the correct area, but not covering all the breast where 23.8%, and 16.6% of
the ROI were not detected.
In total, an acceptable ROI detection (visual score of 2 and 3) was made at first in 95.1%
of the images, and at the moment of detecting the second ROI, 83.3% of the images got
an acceptable ROI detection.
86
4.1.4. Representatives-6
The results obtained from each image of the group Representatives-6 are shown in
table 4.7 and 4.8
Table 4.8 Visual and reliability score obtained from the detection of ROI of each image of Representatives-6 with the ROI previously detected covered.
Table 4.7 presents the results obtained from the ROI detected in the original images of
Representatives-6. The first column indicates the name of each image, at the end it is
presented the average of the visual and reliability score of all the ROI detected in all the
images, and below that is the total of ROI detections who obtained 1, 2 or 3 as visual
score value. The second column contains the visual score of each ROI of each image.
The third score contains the reliability value of each ROI detected in each image.
Table 4.8 presents the results obtained from the ROI detected in the images of
Representatives-6 with the ROI previously detected covered in black color. The first
column indicates the name of each image, at the end it is presented the average of the
visual and reliability score of all the ROI detected in all the images, and below that is the
total of ROI detections who obtained 1, 2 or 3 as visual score value. The second column
contains the visual score of each ROI of each image. The third score contains the
reliability value of each ROI detected in each image.
Representatives-6
Image Visual Result
Reliability Score
TH1_F027 3 1
TH1_F032 2 0.999977
TH3_F049 2 1
TH3_F072 3 1
TH4yTH5_F105 3 0.999990
TH4yTH5_F112 3 1
Average 2.66 0.999994
Total 1: 0
Total 2: 2
Total 3: 4
Representatives-6
Image Visual Result
Reliability Score
TH1_F027 3 0.998733
TH1_F032 2 0.641392
TH3_F049 3 0.998952
TH3_F072 2 0.999984
TH4yTH5_F105 3 0.999679
TH4yTH5_F112 3 0.999103
Average 2.66 0.939640
Total 1: 0
Total 2: 2
Total 3: 4
Table 4.7 Visual and reliability score obtained from the detection of ROI of each original image of Representatives-6.
87
In figure 4.3 are shown three examples, each one with a different thermal classification
and all obtained a visual score of 2 or 3.
Figure 4.4 a) Image TH1_F027, with visual score of 3 in both ROI. b) Image TH3_F049, with visual score of 2 in the first detected ROI, now covered in black, and a visual score of 3 in the
second detected ROI. c) Image TH4yTH5_F105, with visual score of 3 in both ROI.
In the first ROI detection (shown in table 4.7), the average visual score was 2.66 and the
average reliability score was 0.99994, which indicates a good detection of ROI; 66.6% of
the images got the highest value of visual score, it means, the ROI coordinates where
delimiting the breast area correctly. Also 33.3% of the images got ROI coordinates in the
correct area, but not covering all the breast region. In the second ROI detection (shown
in table 4.8), the average visual score was 2.66 and the average reliability score was
0.939640, which is the highest average score in comparison with the ones obtained in the
groups Frontal-123 and CorrectRanges-6, this indicates that images pretty
representatives of each thermographic classification present a better detection of ROI in
both breasts; 66.6% of the images got the highest value of visual score. The total of
second ROI detected in the correct area, but not covering all the breast where 33.3%.
In total, an acceptable ROI detection (visual score of 2 and 3) was made at first in 100%
of the images, and at the moment of detecting the second ROI, also 100% of the images
got an acceptable ROI detection.
88
4.2. ROI manually obtained and Neural Network Pattern Recognition for classification
______________________________________________________________________________
To analyze the feature extraction method and the Neural Network of Pattern Recognition
for classification performance with correctly delimited breasts ROI, it was necessary to
obtained them manually. It was perform the classification with three cases, which were:
I. Frontal-123
II. CorrectRanges-84
III. Representatives-6
Also, there were used different amount of characteristics, to prove if the performance was
better or worst with all or just some of them. So, for each case it was used a different set
of characteristics:
1. Variant 1 (C1, C2, C3, C4, C5, C6, C7)
2. Variant 2 (C1,C2,C3,C4,C5)
3. Variant 3 (C1,C2,C3)
The data obtained from the features extraction of each image of each group was divided
in the next proportions:
Training: 70%
Validation: 15%
Test: 15%
The number of nodes was varied to see which number gave a better performance, there
were used: 18, 20, 22, 24, 26, 28, 30, 40 and 50.
Also, the number of classes were varied, using 2 and 3 classes. When the data was for
training two classes the examples were: healthy and non-healthy groups. When the data
was for training three classes the examples were: healthy, healthy with hyper
vascularization and non-healthy.
The results of the use of different number of nodes with 2 classes, with each case using
variant 1 are shown in the next graphs:
89
Figure 4.5 Graph of the performance of the training and generalization of the group Frontal-123
with different amount of nodes when having 2 classes and variant 1 is used.
Figure 4.6 Graph of the performance of the training and generalization of the group
CorrectRanges-84 with different amount of nodes when having 2 classes and variant 1 is used.
90
Figure 4.7 Graph of the performance of the training and generalization of the group
Representatives-6 with different amount of nodes when having 2 classes and variant 1 is used.
The best performances of training and generalization were obtained from the group
Representatives-6, getting always a 100% of performance with all the different number of
nodes. The results of CorrectRanges-84 in the performance of the generalization, also
called test, were between 60 and 87%, getting its best value when using 50 nodes. In the
group Frontal-123 the best result for generalization was with 20 nodes obtaining 83.3%
of performance, but for training it was with 50 nodes obtaining 91.4% of performance even
that the training percentage was high when using 50 nodes, the generalization
performance was low.
The results of the use of different number of nodes with 3 classes, with each case using
variant 1 are shown in the next graphs:
91
Figure 4.8 Graph of the performance of the training and generalization of the group Frontal-123
with different amount of nodes when having 3 classes and variant 1 is used.
Figure 4.9 Graph of the performance of the training and generalization of the group
CorrectRanges-84 with different amount of nodes when having 3 classes and variant 1 is used.
92
Figure 4.10 Graph of the performance of the training and generalization of the group
Representatives-6 with different amount of nodes when having 3 classes and variant 1 is used.
The best performances of training and generalization were obtained from the group
Representatives-6, getting always a 100% of performance of generalization with all the
different number of nodes, but a lot of variation with the performance of the training. The
results of CorrectRanges-84 in the performance of the generalization were between 38
and 61%, getting its best value when using 20 nodes, but still a low percentage of
performance. In the group Frontal-123 generalization and training performances were
lower compared with the other two groups.
The results of the use of different number of nodes with 2 classes, with each case using
variant 2 are shown in the next graphs:
93
Figure 4.11 Graph of the performance of the training and generalization of the group Frontal-123 with different amount of nodes when having 2 classes and variant 2 is used.
Figure 4.12 Graph of the performance of the training and generalization of the group CorrectRanges-84 with different amount of nodes when having 2 classes and variant 2 is used.
94
Figure 4.13 Graph of the performance of the training and generalization of the group Representatives-6 with different amount of nodes when having 2 classes and variant 2 is used.
The best performances of training and generalization were obtained from the group
Representatives-6, getting always a 100% of performance of training with all the different
number of nodes, and only a bad performance of generalization when using 22 nodes.
The best results of CorrectRanges-84 were when using 50 nodes, getting a performance
of generalization of 75%, and a performance of training of 95%. The group Frontal-123
also showed its better performance when using 50 nodes, getting a performance of
generalization of 75% and training performance of 87.9%. The general results were lower
compared when using variant 1 and 2 classes.
The results of the use of different number of nodes with 3 classes, with each case using
variant 2 are shown in the next graphs:
95
Figure 4.14 Graph of the performance of the training and generalization of the group Frontal-123 with different amount of nodes when having 3 classes and variant 2 is used.
Figure 4.15 Graph of the performance of the training and generalization of the group CorrectRanges-84 with different amount of nodes when having 3 classes and variant 2 is used.
96
Figure 4.16 Graph of the performance of the training and generalization of the group Representatives-6 with different amount of nodes when having 3 classes and variant 2 is used.
The best performances of training and generalization were obtained from the group
Representatives-6, getting 100% of performance of training with 18, 20, 22, 24 and 26
nodes, but had a lot of variation with the performance of the generalization. The best
result of CorrectRanges-84 in the performance of the generalization was 53.8% and a
training performance of 74.1% with 18 nodes, lower than the results obtained with variant
1 and 3 classes. In the group Frontal-123 the best result was with 20 nodes, but
generalization and training performances were lower than 60%.
The results of the use of different number of nodes with 2 classes, with each case using
variant 3 are shown in the next graphs:
97
Figure 4.17 Graph of the performance of the training and generalization of the group Frontal-123 with different amount of nodes when having 2 classes and variant 3 is used.
Figure 4.18 Graph of the performance of the training and generalization of the group CorrectRanges-84 with different amount of nodes when having 2 classes and variant 3 is used.
98
Figure 4.19 Graph of the performance of the training and generalization of the group Representatives-6 with different amount of nodes when having 2 classes and variant 3 is used.
The best performances of training and generalization were obtained from the group
Representatives-6, getting 100% of performance of training with all the different nodes
number but not with 28, its generalization performance was of 100% only with 24, 26, 28
and 50 nodes. The best result of CorrectRanges-84 in the performance of the
generalization was 87.5% and a training performance of 82.5% with 18 nodes. In the
group Frontal-123 the best result was also using 18 nodes, with 83.3% of generalization
and training performance of 72.4%.
The results of the use of different number of nodes with 3 classes, with each case using
variant 3 are shown in the next graphs:
99
Figure 4.20 Graph of the performance of the training and generalization of the group Frontal-123
with different amount of nodes when having 3 classes and variant 3 is used.
Figure 4.21 Graph of the performance of the training and generalization of the group
CorrectRanges-84 with different amount of nodes when having 3 classes and variant 3 is used.
100
Figure 4.22 Graph of the performance of the training and generalization of the group
Representatives-6 with different amount of nodes when having 3 classes and variant 3 is used.
The best performances of training and generalization were obtained from the group
Representatives-6, getting 100% of performance of training when using 18, 20, 24 and
30 nodes, and its generalization performance was of 100% only with 20, 26, and 30
nodes. The results of CorrectRanges-84 and Frontal-123 were both lower than 55% in
the performance of the generalization, and the training performance results were lower
than 65%.
The best cases of generalization and its training performances of each group with each
variant are presented next:
Variant 1 2 classes 3 classes
Case % training % generalization % training % generalization
I.1 70.7 83.3 58.6 55.6
II.1 100 87.5 63.8 61.5
III.1 100 100 100 100 Table 4.9 Best performances of each case with variant 1
Variant 2 2 classes 3 classes
Case % training % generalization % training % generalization
I.2 87.9 75 55.2 55.6
II.2 95 75 74.1 53.8
III.2 100 100 100 100 Table 4.10 Best performances of each case with variant 2
101
Variant 3 2 classes 3 classes
Case % training % generalization % training % generalization
I.3 72.4 83.3 49.4 50
II.3 82.5 87.5 51.7 53.8
III.3 100 100 100 100 Table 4.11 Best performances of each case with variant 3
With the three types of variants, and the 2 and 3 classes, the best performance obtained
was the one of the group Representatives-6. The second group with better performance
in the test with the different variants and classes was CorrectRanges-84, presenting a
better performance when it was divided in two classes. The variant which got the better
training percentage with two and three classes, with all the groups in general, was variant
1, which is the one that contains all the seven characteristics proposed.
To evaluate the algorithm, it was used some criteria for the best cases when the data was
divided in 2 classes, such as Positive Predictive Value (PPV), Negative Predictive Value
(NPV), Sensitivity (SEN) and Specificity (SPC), that are defined as follows (Akobeng,
2007):
𝑃𝑃𝑉 =𝑇𝑃
𝑇𝑃+𝐹𝑃 (19)
𝑁𝑃𝑉 =𝑇𝑁
𝑇𝑁+𝐹𝑁 (20)
𝑆𝐸𝑁 =𝑇𝑃
𝑇𝑃+𝐹𝑁 (21)
𝑆𝑃𝐶 =𝑇𝑁
𝐹𝑃+𝑇𝑁 (22)
In equation 19, PPV is the probability that subjects with a positive test truly have the
disease (Akobeng, 2007); where TP are the true positive cases, people with a disease
correctly identified as sick, and FP are the false positive cases, healthy people incorrectly
identified as sick. In equation 20, NPV is the probability that subjects with a negative test
truly do not have the disease (Akobeng, 2007); where TN are the true negative cases,
healthy people correctly identified as healthy, and FN are the false negative cases, sick
people incorrectly identified as healthy. In equation 21, SEN is the ability of the test to
102
correctly detect sick patients who actually are sick. In equation 22, SPC is the ability of
the test to correctly detect healthy patients who actually do not have a condition.
The values of TP, TN, FP, FN and TOE (total of errors), are obtained from the confusion
matrixes shown in the appendix C.
The results of the evaluation of each case with each variant when the data was divided
in 2 classes are shown next:
Variant 1 Performance measures
Case TP TN FP FN TOE PPV NPV SEN SPC
Test
I.1 5 5 1 1 2 0.83 0.83 0.83 0.83
II.1 4 3 1 0 1 0.80 1.00 1.00 0.75
III.1 1 0 0 0 0 1.00 - 1.00 -
Training
I.1 22 19 8 9 17 0.73 0.68 0.71 0.70
II.1 20 20 0 0 0 1.00 1.00 1.00 1.00
III.1 1 1 0 0 0 1.00 1.00 1.00 1.00 Table 4.12 Performance measures of the best test and training results of each case using
variant 1
Variant 2 Performance measures
Case TP TN FP FN TOE PPV NPV SEN SPC
Test
I.2 5 4 1 2 3 0.83 0.67 0.71 0.80
II.2 4 2 2 0 2 0.67 1.00 1.00 0.50
III.2 1 0 0 0 0 1.00 - 1.00 -
Training
I.2 25 26 3 4 7 0.89 0.87 0.86 0.90
II.2 18 20 1 1 2 0.95 0.95 0.95 0.95
III.2 1 1 0 0 0 1.00 1.00 1.00 1.00
Table 4.13 Performance measures of the best test and training results of each case using variant 2
103
Variant 3 Performance measures
Case TP TN FP FN TOE PPV NPV SEN SPC
Test
I.3 7 3 1 1 2 0.88 0.75 0.88 0.75
II.3 2 5 1 0 1 0.67 1.00 1.00 0.83
III.3 1 0 0 0 0 1.00 - 1.00 -
Training
I.3 19 23 6 10 16 0.76 0.70 0.66 0.79
II.3 17 16 4 3 7 0.81 0.84 0.85 0.80
III.3 1 1 0 0 0 1.00 1.00 1.00 1.00 Table 4.14 Performance measures of the best test and training results of each case using
variant 3
The best results are obtained from the cases II (CorrectRanges-84) and III
(Representatives-6). When using variant 1 the sensitivity for test and training of both
cases is 1, which indicates this is the best variant to work with. The NPV and SPC values
of the case III cannot being calculated for test because the low amount of examples of
this group.
To obtain the performance measures with three classes it was necessary to compare
between two groups, it means that three comparisons were made:
1. Healthy (H) class vs. Healthy with hyper vascularization (HHV) class
2. Healthy (H) class vs. Non-healthy (NH) class
3. Healthy with hyper vascularization (HHV) class vs. Non-healthy N(H) class
The TP values were the ones that were detected as the first class mentioned that actually
belongs to that class, for example, in the first comparison the TP values would be the
ones detected as Healthy that actually belongs to that class, and so on with the other two
comparisons. The same rule applies with the TN, FP, and FN, now the true values would
be the ones detected as the first class mentioned in the comparison that actually belongs
to that class.
The results of the evaluation of each case with each variant when the data was divided
in 3 classes are shown next:
104
H-HHV Performance measures
Case TP TN FP FN TOE PPV NPV SEN SPC
Test
I.1 3 4 1 3 4 0.75 0.57 0.50 0.80
II.1 2 2 1 0 1 0.67 1.00 1.00 0.67
III.1 0 0 0 0 0 - - - -
I.2 1 4 0 3 3 1.00 0.57 0.25 1.00
II.2 5 1 1 1 2 0.83 0.50 0.83 0.50
III.2 0 0 0 0 0 - - - -
I.3 2 1 3 0 3 0.40 1.00 1.00 0.25
II.3 2 3 0 1 1 1.00 0.75 0.67 1.00
III.3 0 0 0 0 0 - - - -
Training
I.1 17 19 4 6 10 0.81 0.76 0.74 0.83
II.1 7 14 1 4 5 0.88 0.78 0.64 0.93
III.1 2 1 0 0 0 1.00 1.00 1.00 1.00
I.2 16 15 5 9 14 0.76 0.63 0.64 0.75
II.2 10 14 1 2 3 0.91 0.88 0.83 0.93
III.2 2 1 0 0 0 1.00 1.00 1.00 1.00
I.3 21 7 6 3 9 0.78 0.70 0.88 0.54
II.3 11 9 5 7 12 0.69 0.56 0.61 0.64
III.3 1 2 0 0 0 1.00 1.00 1.00 1.00 Table 4.15 Performance measures of the best test and training results of each case using
variant 1 and comparing the healthy class with the healthy with hyper vascularization class.
H-NH Performance measures
Case TP TN FP FN TOE PPV NPV SEN SPC
Test
I.1 3 3 1 0 1 0.75 1.00 1.00 0.75
II.1 2 4 0 3 3 1.00 0.57 0.40 1.00
III.1 0 1 0 0 0 - 1.00 - 1.00
I.2 1 5 0 2 2 1.00 0.71 0.33 1.00
II.2 5 1 0 2 2 1.00 0.33 0.71 1.00
III.2 0 1 0 0 0 - 1.00 - 1.00
I.3 2 6 0 2 2 1.00 0.75 0.50 1.00
II.3 2 2 0 1 1 1.00 0.67 0.67 1.00
III.3 0 1 0 0 0 - 1.00 - 1.00
Training
I.1 17 19 4 6 10 0.81 0.76 0.74 0.83
II.1 7 14 1 4 5 0.88 0.78 0.64 0.93
III.1 2 1 0 0 0 1.00 1.00 1.00 1.00
I.2 16 17 1 5 6 0.94 0.77 0.76 0.94
II.2 10 19 1 2 3 0.91 0.90 0.83 0.95
105
III.2 2 1 0 0 0 1.00 1.00 1.00 1.00
I.3 21 15 6 7 13 0.78 0.68 0.75 0.71
II.3 11 10 2 3 5 0.85 0.77 0.79 0.83
III.3 1 1 0 0 0 1.00 1.00 1.00 1.00
Table 4.16 Performance measures of the best test and training results of each case using variant 2 and comparing the healthy class with non-healthy class.
HHV-NH Performance measures
Case TP TN FP FN TOE PPV NPV SEN SPC
Test
I.1 4 3 1 2 3 0.80 0.60 0.67 0.75
II.1 2 4 0 1 1 1.00 0.80 0.67 1.00
III.1 0 1 0 0 0 - 1.00 - 1.00
I.2 4 5 2 1 3 0.67 0.83 0.80 0.71
II.2 1 1 0 2 2 1.00 0.33 0.33 1.00
III.2 0 1 0 0 0 - 1.00 - 1.00
I.3 1 6 2 2 4 0.33 0.75 0.33 0.75
II.3 3 2 4 0 4 0.43 1.00 1.00 0.33
III.3 0 1 0 0 0 - 1.00 - 1.00
Training
I.1 17 19 4 6 10 0.81 0.76 0.74 0.83
II.1 7 14 1 4 5 0.88 0.78 0.64 0.93
III.1 2 1 0 0 0 1.00 1.00 1.00 1.00
I.2 15 17 11 8 19 0.58 0.68 0.65 0.61
II.2 14 19 5 4 9 0.74 0.83 0.78 0.79
III.2 1 1 0 0 0 1.00 1.00 1.00 1.00
I.3 7 15 5 17 22 0.58 0.47 0.29 0.75
II.3 9 10 5 6 11 0.64 0.63 0.60 0.67
III.3 2 1 0 0 0 1.00 1.00 1.00 1.00 Table 4.17 Performance measures of the best test and training results of each case using
variant 3 and comparing the healthy with hyper vascularization class with non-healthy class.
The results shows a lot of variations, and not a good performance in sensibility and
specificity at the same time, when one of those values is high the other is low. The NPV
and PPV seems to get a better result that the others mentioned before. It seems that the
classification networks can see the difference between the healthy class with the others,
but the network gets confused when separating the healthy with hyper vascularization
class with the non-healthy class. In this case, also the Representatives-6 group was the
one with the best performance.
106
4.3. ROI automatically obtained and Neural Network Pattern Recognition for classification
______________________________________________________________________________
After the evaluation of the automatic detection of breasts ROI, and the evaluation of the
method of classification with manually obtained ROI, it was made a test with both
methods. The only group where all the breasts ROI were detected in all the images was
the group Representatives-6, because of the importance of getting balanced data, the
other groups could not be processed with the classification algorithm. So, the case was
just one:
III. Representatives-6
But, the variants remained being three for the only case:
1. Variant 1 (C1, C2, C3, C4, C5, C6, C7)
2. Variant 2 (C1,C2,C3,C4,C5)
3. Variant 3 (C1,C2,C3)
The data division to train, validate ant test the network was the same used in the ROI
detected manually:
Training: 70%
Validation: 15%
Test: 15%
The number of nodes was varied to see which number gave a better performance, there
were used: 18, 20, 22, 24, 26, 28, 30, 40 and 50.
Also, the number of classes were varied, using 2 and 3 classes. When the data was for
training two classes, the examples were from the healthy and non-healthy groups. When
the data was for training three classes, the examples were from the three groups: healthy,
healthy with hyper vascularization and non-healthy.
The results of the use of different number of nodes with 2 classes, with each variant are
shown in the next graphs:
107
Figure 4.23 Graph of the performance of the training and generalization of the group Representatives-6 with different amount of nodes when having 2 classes and variant 1 is used.
Figure 4.24 Graph of the performance of the training and generalization of the group Representatives-6 with different amount of nodes when having 2 classes and variant 2 is used.
108
Figure 4.25 Graph of the performance of the training and generalization of the group Representatives-6 with different amount of nodes when having 2 classes and variant 3 is used.
The best performances of training and generalization were using variant 1, the results are
as good as the ones when using the ROI manually obtained. The training using the three
variants had a 100% of performance the majority of the times. Even the behavior of the
graphs is similar compared to the ones using ROI manually obtained, with variant 1 the
performances are more stable when changing the number of nodes, and with variant 2
and 3 it gets less stable (more ups and downs can be perceived).
The results of the use of different number of nodes with 3 classes, with each variant are
shown in the next graphs:
109
Figure 4.26 Graph of the performance of the training and generalization of the group
Representatives-6 with different amount of nodes when having 3 classes and variant 1 is used.
Figure 4.27 Graph of the performance of the training and generalization of the group
Representatives-6 with different amount of nodes when having 3 classes and variant 2 is used.
110
Figure 4.28 Graph of the performance of the training and generalization of the group
Representatives-6 with different amount of nodes when having 3 classes and variant 3 is used.
The best performances of training and generalization were using variant 1, the results are
as good as the ones when using the ROI manually obtained. The training using the three
variants had a 100% of performance the majority of the times. Something that should be
highlighted is that there were less variations with the generalization performance with
different number of nodes with 3 classes than when using the ROI manually obtained.
The best cases of the group Representative-6 with each variant are presented next:
Variant 1 2 classes 3 classes
Case % training % generalization % training % generalization
III.1 100 100 100 100
Variant 2 2 classes 3 classes
Case % training % generalization % training % generalization
III.2 100 100 100 100
Variant 3 2 classes 3 classes
Case % training % generalization % training % generalization
III.3 100 100 100 100
The best results are the same that the ones using ROI manually obtained, this means,
that at least in with this group it can be assumed that when the visual score is of 2 or 3,
the classification performance is as good when detecting the ROI automatically as when
detecting it manually.
111
4.4. Analyses of Results ______________________________________________________________________________
In an ideal scenario, that is, with thermal images that at first glance the specialist knows
what classification they belong to, the selected characteristics and neural network fulfill
the function of classification, as well as the correct automatic detection of ROI.
The number of examples with which the network is trained is very low, and even so,
classification results greater than 80% are obtained when there are two classes and the
thermographies are not those of an ideal scenario.
The network gets confused when there are 3 classes, and the scenario is not ideal.
According to the matrices of confusion, it is observed that the network confuses the TH3
cases with TH4 and TH5. This also indicates that the network is able to differentiate
normal cases (TH1) from abnormal ones (TH3, TH4 and TH5).
The training percentages, and automatically ROI detection are better in the
CorrectRanges-84 group compared to Frontal-123, which indicates that the control of the
temperature ranges affects the analysis of images.
The percentages of generalization are similar in the groups CorrectRanges-84 and
Frontal-123, this may be due to the fact that in the second group there are more examples
for testing and training, which improves the performance of the network.
It is noteworthy that in this work a better specificity, sensitivity, positive predictive value
and negative predictive value were obtained in the training of the classification than in
other works that uses neural networks for classification (Mookiah, 2012) (Rastghalam,
2016). A better sensitivity and negative predictive value was also obtained in the
execution (test) of the classification than in other studies (Rastghalam, 2016).
112
5. Chapter 5: Conclusions
In this thesis work it was proposed the use of neural network algorithms to detect
automatically breasts, and to classify thermal classes according to the proposed features
extraction. It was found that when the minimum and maximum temperatures of the
thermograms were according to the ones established in the protocol of image acquisition,
the method proposed works correctly, getting a 100% of sensitivity. Also, for the detection
of ROI automatically when the minimum and maximum temperature of each image of the
set is between the same values, the detection is asserted between 95 and 83% of the
times. It can be considered a useful tool for big image sets, because of the high degree
of correct detection and the low cost of time, compared when acquiring the ROI manually.
The suggested characteristics are useful to identify 2 classes (healthy-TH1 and non-
healthy-TH5), but they are not sufficient to identify 3 classes (healthy-TH1, healthy with
hypervascularizations-TH3 and non-healthy-TH5). The training and test performances
are low even when the temperatures of the set of images used are in the same range.
It is needed a more rigorous labeling of the TH3 class, because it is based on
vascularization, but also on calcifications, and it should be noted that the shape of the
calcifications gives information about a benign or a malign tumor. Also, a characteristic
that allows to identify the "healthy" vascularization of the "non-healthy" ones is needed,
as well as benign and malignant tumors.
According to the results obtained with three classes, and their matrices of confusion, it
was observed that the algorithm makes a correct discrimination between what is
considered a normal and abnormal thermogram.
113
5.1. Future Work
______________________________________________________________________________
A greater number of images is necessary to improve the performance of the neural
network of pattern recognition.
A stricter protocol to acquire the thermographies is needed to improve classification
results and automatic identification of ROI. It can be improved by limiting the number of
people in the room where the thermograms are acquired. Also, using a thermometer to
be sure that when the images are taken the room temperature coincides with the one that
the system of air conditioner shows. To get always the same position of the patients for
frontal and lateral images, some marks in the floor can be made, pointing each one of the
positions. Finally, the menstrual cycle should be considered, it would be a variable to not
take in count if all the patients were in a same stage of the cycle when their thermograms
are taken.
The deep learning can be used directly to extract automatically features and classify the
thermograms.
The addition of qualitative variables, as age of the patient, clinical history, etc., to the
classification algorithm.
114
Appendix
A. Pseudocode of characteristics ______________________________________________________________________________
Process Characteristic_C1 Read ROI_left, ROI_right
LBP_left←extractLBPFeatures(ROI_left)
LBP_ right ←extractLBPFeatures(ROI_right)
Left_vs_Right←(LBP_left-LBP_ right).^2
EndProcess
______________________________________________________________________________
Algorithm Characteristic_C2
Read ROI_right, ROI_left
GausIfanFilter←[0 1 2 1 0;1 3 5 3 1;2 5 9 5 2;1 3 5 3 1;0 1 2 1 0]*(1/40)
G1←imfilter(ROI_right, GausIfanFilter)
G2←imfilter(ROI_leftt, GausIfanFilter)
[row,column] ←size(G1)
i←1
j←1
k←1
G← [G1,G2]
Repeat
G←G(k)
While i<row Do
While j<column Do
If G(i,j)<220 & G(i,j)>255 Then
G(i,j) ←1
IfNot
G(i,j) ←0
End If
j←j+1
End While
j←1
i←i+1
End While
k←k+1
Until k>2
r←row/10
c←column/10
ii←r
jj←c
i←1
j←1
k←1
115
l←1
Repeat
G←G(k)
While r<row Do
While c<column Do
temp(l) ←G(i:ii,j:jj)
l←l+1
j←j+c
jj←jj+c
End While
j←1
jj←c
i←i+r
ii←ii+r
End While
G100(k) ←temp
k←k+1
Until k>2
k←1
n←1
Repeat
G100←G100(k)
While n<=100 Do
c←0
While i<r Do
While j<c Do
If G100(n)(i,j)==0 Then
c←c+1
End If
j←j+1
End While
j←1
i←i+1
End While
G100_numBlackPixels(n) ←c
n←n+1
End While
Until k>2
k←1
i←1
j←10
n←1
Repeat
G100_numBlackPixels←G100_numBlackPixels(k)
While n<10 Do
temp(n) ←G100_numBlackPixels(i:j)
116
n←n+1
i←j+1
j←j+10
End While
G10(k) ←temp
k←k+1
Until k>2
G_compare←abs(G10(1)-G10(2))
EndAlgorithm
______________________________________________________________________________
Algorithm Characteristic_C3
Read ROI_right, ROI_left
histogram_right←imhist(ROI_right)
histogram_left←imhist(ROI_left)
HistComp←corrcoef(histogram_right,histogram_left)
EndAlgorithm
______________________________________________________________________________
Algorithm Characteristic _C4
Read ROI_right, ROI_left
histogram_right←imhist(ROI_right)
histogram_left←imhist(ROI_left)
Int_HigherFrec_right←max(histogram_right)
Int_HigherFrec_left←max(histogram_left)
Dif_IntHigherFrec←abs(Int_HigherFrec_right - Int_HigherFrec_left)
EndAlgorithm
______________________________________________________________________________
Algorithm Characteristic _C5
Read ROI_right, ROI_left
DesvEst_right←mean(std(ROI_right))
DesvEst_left←mean(std(ROI_left))
EndAlgorithm
______________________________________________________________________________
Algorithm Characteristic_C6
Read ROI_right, ROI_left
GaussianFilter=[0 1 2 1 0;1 3 5 3 1;2 5 9 5 2;1 3 5 3 1;0 1 2 1 0]*(1/80)
G_right←imfilter(ROI_right,GaussianFilter)
G_left←imfilter(ROI_left,GaussianFilter)
[M,N] ←size(G_right)
F_right←fft2(G_right)/sqrt(M*N)
F_left←fft2(G_left)/sqrt(M*N)
F_right←log(1 + abs(ffshift(F_right)))
F_left←log(1 + abs(ffshift(F_left)))
k←1
i←1
117
j←1
Repeat
If k==1 Then
F←F_right
IfNot
F←F_left
End If
While i<M Do
While j<N Do
If F(i,j)<6.5 Then
F(i,j) ←0
End If
j←j+1
End While
j←1
i←i+1
End While
If k==1 Then
F_right←F
IfNot
F_left←F
End If
k←k+1
Until k>2
C←corrcoef(F_right,F_left)
EndAlgorithm
______________________________________________________________________________
Algorithm Characteristic _C7
Read ROI_right, ROI_left
GaussianFilter=[0 1 2 1 0;1 3 5 3 1;2 5 9 5 2;1 3 5 3 1;0 1 2 1 0]*(1/80)
G_right←imfilter(ROI_right,GaussianFilter)
G_left←imfilter(ROI_left,GaussianFilter)
[M,N] ←size(G_right)
F_rightVfft2(G_right)/sqrt(M*N)
F_left←fft2(G_left)/sqrt(M*N)
F_right←log(1 + abs(ffshift(F_right)))
F_left←log(1 + abs(ffshift(F_left)))
k←1
i←1
j←1
Repeat
If k==1 Then
F←F_right
IfNot
F←F_left
End If
118
While i<M Do
While j<N Do
If F(i,j)<3.2 Then
F(i,j) ←0
End If
j←j+1
End While
j←1
i←i+1
End While
If k==1 Then
F_right←F
IfNot
F_left←F
End If
k←k+1
Until k>2
k←1
[r,c] ←Find(F_right==max(max(F_right)))
Repeat
If k==1 Then
F←F_right
IfNot
F←F_left
End If
a←F(2:r,2:c)
b←F(2:r,2:N)
C←corrcoef(a,fliplr(b))
If k==1 Then
C_right←C
IfNot
C_left←C
End If
k←k+1
Until k>2
EndAlgorithm
______________________________________________________________________________
119
B. ROI results using different pallets
In section 4.1 is explained that a visual and a reliability score were given to each image,
and the way it was assign. As first test 10 images from the group ROI-testing were used
to verify which pallet gave a better detection result.
Pallet1
Image Visual Result
Reliability Score
1 3 0.999765
2 2 0.870618
3 3 1
4 2 0.999816
5 2 0.99984
6 3 1
7 2 0.999854
8 2 1
9 2 0.999997
10 2 1
Average= 2.3 0.986989
Total 1: 0
Total 2: 7
Total 3: 3
Pallet1
Image Visual Result
Reliability Score
1 3 0.999988
2 2 0.511646
3 2 1
4 3 0.999991
5 2 0.999984
6 3 1
7 2 0.999937
8 3 1
9 3 0.999973
10 3 1
Average= 2.6 0.9511519
Total 1: 0
Total 2: 4
Total 3: 6
Table B.1 Visual and reliability score obtained from the detection of ROI of each image of the 10 selected images from ROI-testing with pallet 1 color code with the ROI previously detected covered.
Table B.2 Visual and reliability score obtained from the detection of ROI of each original image of the 10 selected images from ROI-testing with pallet 1 color code.
Pallet9
Image Visual Result
Reliability Score
1 2 0.999999
2 3 1
3 1 0.998749
4 2 0.999999
5 3 1
6 2 0.999509
7 3 1
8 2 0.999999
9 2 0.999995
Pallet9
Image Visual Result
Reliability Score
1 2 0.999999
2 3 0.999999
3 1 0.999121
4 2 0.999997
5 2 0.999999
6 2 0.999982
7 2 0.999999
8 3 1
9 2 0.999994
120
Table B.4 Visual and reliability score obtained from the detection of ROI of each original image of the 10 selected images from ROI-testing with pallet 9 color code.
Table B.6 Visual and reliability score obtained from the detection of ROI of each image of the 10 selected images from ROI-testing with pallet 9 color code with the ROI previously detected covered.
The lowest average of visual score was obtained with pallet9 (2.3 and 2.6), then with
pallet 1 (2.1), and finally the best score was obtained when using pallet 4 (2.7).
10 1 0.999955
Average 2.1 0.9998205
Total 1: 2
Total 2: 5
Total 3: 3
10 2 0.999701
Average 2.1 0.9998791
Total 1: 1
Total 2: 7
Total 3: 2
Pallet4
Image Visual Result
Reliability Score
1 3 0.999998
2 2 1
3 1 0
4 3 0.997301
5 3 1
6 3 0.997981
7 3 0.999999
8 3 0.999987
9 3 0.999994
10 3 0.999984
Average 2.7 0.88836067
Total 1: 1
Total 2: 1
Total 3: 8
Table B.3 Visual and reliability score obtained from the detection of ROI of each image of the 10 selected images from ROI-testing with pallet 9 color code with the ROI previously detected covered.
Pallet4
Image Visual Result
Reliability Score
1 3 1
2 3 1
3 2 0.916612
4 3 0.999632
5 2 1
6 3 0.999995
7 2 1
8 2 0.999583
9 3 0.999998
10 3 1
Average 2.6 0.991582
Total 1: 0
Total 2: 4
Total 3: 6
Table B.5 Visual and reliability score obtained from the detection of ROI of each original image of the 10 selected images from ROI-testing with pallet 9 color code.
121
C. Confusion matrixes
The confusion matrixes of the best cases of performance when the data was divided in
two classes are presented next:
Figure C.1 Confusion matrix of group Frontal-123 using Variant 1
and 18 nodes
Figure C.2 Confusion matrix of group CorrectRanges-84 using
Variant 1 and 18 nodes
122
Figure C.2 Confusion matrix of group Representatives-6 using
Variant 1 and 28 nodes
Figure C.1 Confusion matrix of group Frontal-123 using Variant 2
and 50 nodes
123
Figure C.4 Confusion matrix of group CorrectRanges-84 using
variant 2 and 50 nodes
Figure C.3 Confusion matrix of group Representatives-6 using
variant 2 and 18 nodes
124
0
Figure C.5 Confusion matrix of group Frontal-123 using variant 3
and 18 nodes
Figure C.6 Confusion matrix of group CorrectRanges-84 using
variant 3 and 18 nodes
125
Figure C.7 Confusion matrix of group Representatives-6 using variant 3 and 24 nodes
126
Figures
Figure 2.1 Electromagnetic wave (Radiansa_Consulting©, 2017). ..................................... 8
Figure 2.2 Espectro electromagnético (Mini_Physics©, 2017). ............................................ 9
Figure 2.3 Planck Curve (Alice's_Astro_Info©, 2017). ......................................................... 18 Figure 2.4 Thermogram of a bat (Nudelman, 1980). ............................................................ 22
Figure 2.5 Elements of an IR camera (© Flir® Systems, 2017) ......................................... 23 Figure 2.6 Thermogram of breast (Nudelman, 1980). .......................................................... 24
Figure 2.7 Left: TH1 image. Right: TH5 image (TH-CEPREC, 2008-2009). .................... 26 Figure 2.8 Left: original image. Right: Fourier transform image. ........................................ 28
Figure 2.9 Left: original image. Right: segmented image with threshold range [220-255] (TH-CEPREC, 2008-2009). ...................................................................................................... 30
Figure 2.10 Circularly symmetric neighbor sets for different radius a) R=1 b) R=1.5 c) R=2 (Eghtesad & Amirani, 2013). ............................................................................................ 31
Figure 2.11 Local Binary Pattern process (García-Olalla, Alegre, Fernández-Robles, & García-Ordás, 2012) .................................................................................................................. 31
Figure 2.12. a) TH1 image with its LBP histogram b) TH5 image with its LBP histogram. Bars in blue belongs to the LBP values of the right breast, and yellow bars belongs to the left breast LBP values (TH-CEPREC, 2008-2009). ....................................................... 33
Figure 2.13 TH5 image and the histogram of each breast. ................................................. 34
Figure 2.14 Neural network architecture (Vadivambal & Digvir, 2016) ............................. 35
Figure 2.15 Example of Convolutional Neural Network arrangement of 3 dimensions (MathWorks®, Object Detection Using Deep Learning, 2017) ........................................... 37
Figure 2.16 Performing a convolutional operation (Apple_Inc.®, 2016) ........................... 38
Figure 2.17 Max pooling in CNN (MathWorks®, 2017) ........................................................ 38
Figure 2.18 Blocks diagram of the R-CNN ............................................................................. 39
Figure 2.19 General scheme of pattern recognition network architecture (MathWorks®, 2017) ............................................................................................................................................ 41 Figure 3.1 a) Frontal Image, b) right lateral image and c) left lateral image. (TH-CEPREC, 2008-2009) ............................................................................................................... 44
Figure 3.2 Classification of thermograms by thermo biological criteria: a) TH1, b) TH2, c) TH3, d) TH4, e) TH5 (TH-CEPREC, 2008-2009) ............................................................ 46 Figure 3.3 . a) Areas of the body with the highest temperature (28.5 ºC) b) The blue zones represent the temperature of the room, which is the minimum temperature recorded (19.8 ºC). (TH-CEPREC, 2008-2009) .................................................................... 50
Figure 3.4 a) Minimum temperature distribution of each one of the 123 images ............ 53
Figure 3.5 Image of healthy person with mastectomy in left breast, visualized in the 10 different color palettes (TH-CEPREC, 2008-2009). .............................................................. 54 Figure 3.6 Image of patient with cancer in left breast, the color code of palette 4 is used (TH-CEPREC, 2008-2009). ...................................................................................................... 55
Figure 3.7 Method used to select the ROIs and obtain their coordinates (TH-CEPREC, 2008-2009). ................................................................................................................................. 56
Figure 3.8 Image of patient with cancer in left breast, visualized in grayscale (TH-CEPREC, 2008-2009). .............................................................................................................. 57
Figure 3.9 Stop signs detection by the use of R-CNN (MathWorks®, 2017). .................. 58
127
Figure 3.10 . a) ROI detection with reliability value of 1 b) ROI detection with reliability value of 0.832 (TH-CEPREC, 2008-2009). ............................................................................ 59 Figure 3.11 Left: original image. Right: image with previously detect ROI covered. ....... 60
Figure 3.12 a) Thermogram of a healthy woman (TH1) b) Thermogram of a woman with cancer on her right breast (TH5). (TH-CEPREC, 2008-2009). ........................................... 61 Figure 3.13 Graphical representation of how the correlation value of LBP between breasts is obtained. .................................................................................................................... 63
Figure 3.14 A) Original image B) Filtered image C) Binary image (TH-CEPREC, 2008-2009). ........................................................................................................................................... 63 Figure 3.15 a) Right breast ROI with 100 subdivisions c) Right breast ROI with 100 subdivisions ................................................................................................................................. 64
Figure 3.16 Graphic representation of how matrices a and b are compared. .................. 64
Figure 3.17 Graphical representation of the obtaining of the correlation coefficient of the histograms of the left and right breast (TH-CEPREC, 2008-2009). ................................... 65
Figure 3.18 Graphical representation of the obtaining of the correlation coefficient of the Fourier transforms of the left and right breast (TH-CEPREC, 2008-2009). ...................... 67
Figure 3.19 a) ROI (right breast) b) Fourier Transform c) Filtered Fourier transform (TH-CEPREC, 2008-2009). .............................................................................................................. 68
Figure 3.20 Quadrant division of the transform. .................................................................... 68
Figure 3.21 Graphical representation of the obtaining of the correlation coefficient of I-II quadrants of the Fourier transform. ......................................................................................... 69 Figure 3.22 Sigmoid function. .................................................................................................. 71
Figure 4.1 a) Image with a visual score value of 3, and a reliability score of 1 b) Image with a visual score of 2, and a reliability score of 0.999458 c) Image with a visual score of 1, and a reliability score of 0.930862. ................................................................................. 75
Figure 4.2 a) Image TH1_F001, with visual score of 3 in both ROI. b) Image TH3_F048, with visual score of 3 in both ROI. c) Image TH4yTH5_F112, with visual score of 3 in both ROI. ..................................................................................................................................... 81 Figure 4.3 a) Image TH1_F009, with visual score of 3 in both ROI. b) Image TH3_F044, with visual score of 3 in both ROI. c) Image TH4yTH5_F120, with visual score of 3 in both ROI. ..................................................................................................................................... 85
Figure 4.4 a) Image TH1_F027, with visual score of 3 in both ROI. b) Image TH3_F049, with visual score of 2 in the first detected ROI, now covered in black, and a visual score of 3 in the second detected ROI. c) Image TH4yTH5_F105, with visual score of 3 in both ROI. ..................................................................................................................................... 87 Figure 4.5 Graph of the performance of the training and generalization of the group Frontal-123 with different amount of nodes when having 2 classes and variant 1 is used........................................................................................................................................................ 89
Figure 4.6 Graph of the performance of the training and generalization of the group CorrectRanges-84 with different amount of nodes when having 2 classes and variant 1 is used. ......................................................................................................................................... 89
Figure 4.7 Graph of the performance of the training and generalization of the group Representatives-6 with different amount of nodes when having 2 classes and variant 1 is used. ......................................................................................................................................... 90
128
Figure 4.8 Graph of the performance of the training and generalization of the group Frontal-123 with different amount of nodes when having 3 classes and variant 1 is used........................................................................................................................................................ 91
Figure 4.9 Graph of the performance of the training and generalization of the group CorrectRanges-84 with different amount of nodes when having 3 classes and variant 1 is used. ......................................................................................................................................... 91 Figure 4.10 Graph of the performance of the training and generalization of the group Representatives-6 with different amount of nodes when having 3 classes and variant 1 is used. ......................................................................................................................................... 92
Figure 4.11 Graph of the performance of the training and generalization of the group Frontal-123 with different amount of nodes when having 2 classes and variant 2 is used........................................................................................................................................................ 93
Figure 4.12 Graph of the performance of the training and generalization of the group CorrectRanges-84 with different amount of nodes when having 2 classes and variant 2 is used. ......................................................................................................................................... 93
Figure 4.13 Graph of the performance of the training and generalization of the group Representatives-6 with different amount of nodes when having 2 classes and variant 2 is used. ......................................................................................................................................... 94
Figure 4.14 Graph of the performance of the training and generalization of the group Frontal-123 with different amount of nodes when having 3 classes and variant 2 is used........................................................................................................................................................ 95
Figure 4.15 Graph of the performance of the training and generalization of the group CorrectRanges-84 with different amount of nodes when having 3 classes and variant 2 is used. ......................................................................................................................................... 95
Figure 4.16 Graph of the performance of the training and generalization of the group Representatives-6 with different amount of nodes when having 3 classes and variant 2 is used. ......................................................................................................................................... 96 Figure 4.17 Graph of the performance of the training and generalization of the group Frontal-123 with different amount of nodes when having 2 classes and variant 3 is used........................................................................................................................................................ 97
Figure 4.18 Graph of the performance of the training and generalization of the group CorrectRanges-84 with different amount of nodes when having 2 classes and variant 3 is used. ......................................................................................................................................... 97
Figure 4.19 Graph of the performance of the training and generalization of the group Representatives-6 with different amount of nodes when having 2 classes and variant 3 is used. ......................................................................................................................................... 98
Figure 4.20 Graph of the performance of the training and generalization of the group Frontal-123 with different amount of nodes when having 3 classes and variant 3 is used........................................................................................................................................................ 99
Figure 4.21 Graph of the performance of the training and generalization of the group CorrectRanges-84 with different amount of nodes when having 3 classes and variant 3 is used. ......................................................................................................................................... 99
Figure 4.22 Graph of the performance of the training and generalization of the group Representatives-6 with different amount of nodes when having 3 classes and variant 3 is used. ....................................................................................................................................... 100
129
Figure 4.23 Graph of the performance of the training and generalization of the group Representatives-6 with different amount of nodes when having 2 classes and variant 1 is used. ....................................................................................................................................... 107
Figure 4.24 Graph of the performance of the training and generalization of the group Representatives-6 with different amount of nodes when having 2 classes and variant 2 is used. ....................................................................................................................................... 107
Figure 4.25 Graph of the performance of the training and generalization of the group Representatives-6 with different amount of nodes when having 2 classes and variant 3 is used. ....................................................................................................................................... 108 Figure 4.26 Graph of the performance of the training and generalization of the group Representatives-6 with different amount of nodes when having 3 classes and variant 1 is used. ....................................................................................................................................... 109
Figure 4.27 Graph of the performance of the training and generalization of the group Representatives-6 with different amount of nodes when having 3 classes and variant 2 is used. ....................................................................................................................................... 109
Figure 4.28 Graph of the performance of the training and generalization of the group Representatives-6 with different amount of nodes when having 3 classes and variant 3 is used. ....................................................................................................................................... 110 Figure C.1 Confusion matrix of group CorrectRanges-84 using Variant 1 and 18 nodes..................................................................................................................................................... 121
Figure C.2 Confusion matrix of group Frontal-123 using Variant 1 and 18 nodes ........ 121
Figure C.3 Confusion matrix of group Frontal-123 using Variant 2 and 50 nodes ........ 122 Figure C.4 Confusion matrix of group Representatives-6 using Variant 1 and 28 nodes..................................................................................................................................................... 122
Figure C.5 Confusion matrix of group Representatives-6 using variant 2 and 18 nodes..................................................................................................................................................... 123
Figure C.6 Confusion matrix of group CorrectRanges-84 using variant 2 and 50 nodes..................................................................................................................................................... 123
Figure C.7 Confusion matrix of group Frontal-123 using variant 3 and 18 nodes ......... 124
Figure C.8 Confusion matrix of group CorrectRanges-84 using variant 3 and 18 nodes..................................................................................................................................................... 124
Figure C.9 Confusion matrix of group Representatives-6 using variant 3 and 24 nodes..................................................................................................................................................... 125
130
Tables
Table 1.1 Categories of mammogram findings (Mandelblatt, et al., 2009) ......................... 2
Table 2.1 Targets that represents two classes. .................................................................... 41
Table 2.2 Targets that represents three classes. .................................................................. 41 Table 2.3 Example of assignment of inputs and outputs. .................................................... 42
Table 3.1 Number of people belonging to each classification ............................................ 48 Table 3.2 Total of images with same maximum temperature value and breakdown of number of images according to the class to which they belong. ........................................ 51
Table 3.3 Total of images with same minimum temperature value and breakdown of number of images according to the class to which they belong. ........................................ 52 Table 3.4 Comparison of results of characteristics C3, C4 and C6 when an image is visually easy to identify to which thermal classification belongs and when it is confusing........................................................................................................................................................ 70
Table 4.1Visual and reliability score obtained from the detection of ROI of each original image of ROI-testing. ................................................................................................................. 76
Table 4.2 Visual and reliability score obtained from the detection of ROI of each image of ROI-testing with the ROI previously detected covered. ................................................... 76 Table 4.3 Visual and reliability score obtained from the detection of ROI of each original image of Frontal-123. ................................................................................................................ 80 Table 4.4 Visual and reliability score obtained from the detection of ROI of each image of Frontal-123 with the ROI previously detected covered. .................................................. 80
Table 4.5 Visual and reliability score obtained from the detection of ROI of each original image of CorrectRanges-84. .................................................................................................... 84
Table 4.6 Visual and reliability score obtained from the detection of ROI of each image of CorrectRanges-84 with the ROI previously detected covered. ...................................... 84
Table 4.7 Visual and reliability score obtained from the detection of ROI of each original image of Representatives-6...................................................................................................... 86 Table 4.8 Visual and reliability score obtained from the detection of ROI of each image of Representatives-6 with the ROI previously detected covered. ...................................... 86
Table 4.9 Best performances of each case with variant 1 ................................................. 100
Table 4.10 Best performances of each case with variant 2 .............................................. 100
Table 4.11 Best performances of each case with variant 3 .............................................. 101
Table 4.12 Performance measures of the best test and training results of each case using variant 1........................................................................................................................... 102
Table 4.13 Performance measures of the best test and training results of each case using variant 2........................................................................................................................... 102
Table 4.14 Performance measures of the best test and training results of each case using variant 3........................................................................................................................... 103 Table 4.15 Performance measures of the best test and training results of each case using variant 1 and comparing the healthy class with the healthy with hyper vascularization class. ............................................................................................................... 104
Table 4.16 Performance measures of the best test and training results of each case using variant 2 and comparing the healthy class with non-healthy class. ...................... 105
131
Table 4.17 Performance measures of the best test and training results of each case using variant 3 and comparing the healthy with hyper vascularization class with non-healthy class. ............................................................................................................................ 105
Table B.1 Visual and reliability score obtained from the detection of ROI of each image of the 10 selected images from ROI-testing with pallet 1 color code with the ROI previously detected covered. .................................................................................................. 119
Table B.2 Visual and reliability score obtained from the detection of ROI of each original image of the 10 selected images from ROI-testing with pallet 1 color code. .................. 119
Table B.3 Visual and reliability score obtained from the detection of ROI of each image of the 10 selected images from ROI-testing with pallet 9 color code with the ROI previously detected covered. .................................................................................................. 120
Table B.4 Visual and reliability score obtained from the detection of ROI of each original image of the 10 selected images from ROI-testing with pallet 9 color code. .................. 120
Table B.5 Visual and reliability score obtained from the detection of ROI of each original image of the 10 selected images from ROI-testing with pallet 9 color code. .................. 120
Table B.6 Visual and reliability score obtained from the detection of ROI of each image of the 10 selected images from ROI-testing with pallet 9 color code with the ROI previously detected covered. .................................................................................................. 120
132
References
© Flir® Systems, I. (2017). Staring Focal Plane Arrays (FPAs). Retrieved on November 22 of
2017 from: http://www.flir.com/cores/display/?id=51939
© Flir® Systems, I. (2017). What is Infrared? Retrieved on November 22 of 2017 from:
http://www.flir.com/about/display/?id=41536
Agudelo, N. V. (2008). Radiación infrarroja y termografía. Antioquia, Colombia.: Escuela de
Ingeniería de Antioquia-Instituto de Ciencias de la Salud.
Akobeng, A. K. (2007). Understanding diagnostic tests 1: sensitivity, specificity and predictive
values. Acta paediatrica, 96(3), 338-341.
Alice's_Astro_Info©. (2017). AstroInfo. Retrieved on November 11 of 2017 from:
http://www.alicesastroinfo.com/
Apple_Inc.®. (2016). Developer. Retrieved from Apple Inc. on August 1 of 2017 from:
https://developer.apple.com/library/content/documentation/Performance/Conceptual/vIm
age/ConvolutionOperations/ConvolutionOperations.html
Arora, N. M. (2008). Effectiveness of a noninvasive digital infrared thermal imaging system in
the detection of breast cancer. The American Journal of Surgery, 196(4), 523-526.
Barreto, J. A. (2008). Radiación de un Cuerpo Negro. Ley de Emisión de Planck. Una Breve
Introducción. Instituto de Astronomía-UNAM.
Bengio, Y., Courville, A., & Vincent, P. (2013). Representation Learning: A Review and New
Perspectives. IEEE transactions on pattern analysis and machine intelligence, 1798-
1828.
Borchartt, T. B., Conci, A., Lima, R. C., Resmini, R., & Sanchez, A. (2013). Breast
thermography from an image processing viewpoint: A survey. Signal Processing, 93(10),
2785-2803.
Boundless. (2017, August). Homeostasis: Thermoregulation. Retrieved from Boundless.com on
August 1 of 2017: https://www.boundless.com/biology/textbooks/boundless-biology-
textbook/the-animal-body-basic-form-and-function-33/homeostasis-194/homeostasis-
thermoregulation-743-11974/
Bracewell, R. N. (2000). Cap 2. Groundwork. In The Fourier Transform and its Applications.
Third Edition. (pp. pp. 11-13). Mc Graw Hill.
Breastcancer.org. (2017, May 9). Breastcancer.org. Retrieved from Análisis para detectar el
cáncer de mama: detección, diagnóstico y control on May 9 of 2017:
http://www.breastcancer.org/es/sintomas/analisis/tipos
BreastCancer.org. (2017, March 10). U.S. Breast Cancer Statistics. Retrieved from
BreastCancer.org on March 10 of 2017:
http://www.breastcancer.org/symptoms/understand_bc/statistics
Britz, D. (2015, July). WILDML. Retrieved from Understanding Convolutional Neural Networks
for NLP on July 25 of 2017: http://www.wildml.com/2015/11/understanding-
convolutional-neural-networks-for-nlp/
Callen, H. B. (1998). 1. The Problem and the Postulates. In Thermodynamics and an
Introduction to Thermostatistics. (pp. 5-9). Canadá: John Wiley & Sons, Inc.
Cardona, M., & al., e. (2008). Max Planck, a Conservative Revolutionary. Il Nuovo Saggiatore
24, 39-54.
133
Coleman, M. P., Quaresma, M., Berrino, F., Lutz, J. M., De Angelis, R., Capocaccia, R., & ... &
Micheli, A. (2008). Cancer survival in five continents: a worldwide population-based
study (CONCORD). The lancet oncology, 9(8), 730-756.
de Prada Pérez de Azpeitia, F. I. (2016). La termografía infrarroja: un sorprendente recurso para
la enseñanza de la física y la química. Revista Eureka sobre Enseñanza y Divulgación de
las Ciencias.
Deng, L., & Yu, D. (2014). Deep Learning: Methods and Applications. Foundations and
Trends® in Signal Processing, 197-387.
Eghtesad, H. R., & Amirani, M. C. (2013). Texture Classification with Local Binary Pattern
Based on Continues Wavelet Transformation. International Journal of Advanced
Research in Electrical, Electronics and Instrumentation Engineering.
FDA. (2011, June 2). Breast Cancer Screening - Thermography is Not an Alternative to
Mammography: FDA Safety Communication. Retrieved from U. S. Food & Drug
Administration on June 2 of 2017:
https://www.fda.gov/MedicalDevices/Safety/AlertsandNotices/ucm257259.htm
Felicia Marie Knaul, P., Gustavo Nigenda, P., Rafael Lozano, M. M., Héctor Arreola-Ornelas,
M. e., Ana Langer, M., & Julio Frenk, P. (2009, January). Salud Pública de México.
Retrieved from Cáncer de mama en México: una prioridad apremiante on January 15 of
2017: http://www.scielosp.org/scielo.php?pid=S0036-
36342009000800026&script=sci_arttext&tlng=es
Fernández, J. L. (2017). Temperatura. Retrieved from FisicaLab on November 14 of 2017 from:
https://www.fisicalab.com/apartado/temperatura#contenidos
Fisher, R., Perkins, S., Walker, A., & Wolfart, E. (2003). Fourier Transform. Retrieved from
Informatics Homepages Server on January 15 of 2017:
http://homepages.inf.ed.ac.uk/rbf/HIPR2/fourier.htm
Flir®. (2017). Veterinary applications of thermography. Retrieved on November 10 of 2017
from:
http://www.flir.com/uploadedFiles/Thermography/MMC/Brochures/T820340/T820340_
EN.pdf
Fluke. (2009). Introducción a los Principios de la Termografía. . Fluke Corporation + The Snell
Group. American Technical Publishers.
García-Olalla, O., Alegre, E., Fernández-Robles, L., & García-Ordás, M. T. (2012). Vitality
assessment of boar sperm using an adaptive LBP based on oriented deviation. Asian
Conference on Computer Vision. Springer, Berlin, Heidelberg., 61-72.
Gonzalez, F. J. (2007). Infrared imager requirements for breast cancer detection. In Engineering
in Medicine and Biology Society. 29th Annual International Conference of the IEEE, pp.
3312-3314.
Gonzalez, R. C., & Woods, R. E. (1992). Cap. 4 "Image Enhancement in the Frequency
Domain". In G. &. Woods, Digital Image Processing (pp. pp. 154-161). New Jersey:
Prentice Hall.
Gurney, K. (1997). An introduction to neural networks. London: Routledge.
Gutierrez Delgado, F. (2011). Feasibility of new-generation infrared screening for breast cancer
in rural communities. pp. 92-6.
HAWLICK, R. M. (1979). Statistical and structural approaches to texture. Proceedings of the
IEEE, vol. 67, no. 5, pp. 786–804.
134
Hecht, E. (2002). Chapter 2. Wave Motion. In Optics. Fourth edition. (pp. 10-13). San
Francisco: Addison Wesley.
Hecht, E. (2002). Chapter 3. The Electromagnetic-Photon Spectrum. In Optics. Fourth edition
(pp. 50-79). San Francisco: Addison Wesley.
Hobbins, W. (1987). In Abnormal thermogram-significance in breast cancer (p. pp.12:337).
Interamer, J Rad.
Huang, Y., Kangas, L. J., & Rasco, B. A. (2007). Applications of artificial neural networks
(ANN) in food science and nutrition. Critical Reviews in Food Science and Nutrition,
47(2):113-126.
Jahne, B. (1991). Digital Image Processing: concepts, algorithms and scientific applications.
New York: Springer-Verlag.
Jähne, B. (1991). Digital Image Processing: concepts, algorithms and scientific applications.
New York: Springer-Verlag.
Jiji Latif, M. (2009). Heat conduction. . Berlin and Heidelberg: Springer-Verlag.
Kennedy, D. A. (2009). "A comparative review of thermography as a breast cancer screening
technique." . Integrative cancer therapies., 9-16.
Khurana, I. (2014). Medical Physiology for Undergraduate Students - E-book. Elsevier Health
Sciences.
Koay, J., Herry, C., & Frize, M. (2004). Analysis of breast thermography with an artificial neural
network. Engineering in Medicine and Biology Society. IEMBS '04. 26th Annual
International Conference of the IEEE, Vol. 1, pp. 1159-1162.
Krizhevsky, A., & Hinton, G. (2009). Learning multiple layers of features from tiny images.
Lau, C. (1991). Neural Networks: Theoretical Foundations and Analysis. New York: IEEE.
Malacara, D. (2004). XIX. Radiación del cuerpo negro. In Óptica Básica (pp. 431-437). México,
D. F.: Fondo de Cultura Económica.
Mandelblatt, J. S., Cronin, K. A., Bailey, S., Berry, D. A., De Koning, H. J., & Draisma, G.
(2009). Effects of Mammography Screening Under Different Screening Schedules:
Model Estimates of Potential Benefits and Harms. Annals of internal medicine, 738-747.
MathWorks®. (2017). Classify Patterns with a Shallow Neural Network. Retrieved on
November 23 of 2017 from: https://www.mathworks.com/help/nnet/gs/classify-patterns-
with-a-neural-network.html#f9-48630
MathWorks®. (2017). corrcoef. Retrieved on November 19 of 2017 from:
https://www.mathworks.com/help/matlab/ref/corrcoef.html?requestedDomain=www.mat
hworks.com
MathWorks®. (2017). Deep Learning Using Bayesian Optimization. Retrieved on October 31 of
2017 from: https://www.mathworks.com/help/nnet/examples/deep-learning-using-
bayesian-optimization.html?searchHighlight=L2Regularization&s_tid=doc_srchtitle
MathWorks®. (2017). Object Detection Using Deep Learning. Retrieved from MathWorks on
July 16 of 2017 from: https://www.mathworks.com/help/vision/examples/object-
detection-using-deep-learning.html?requestedDomain=www.mathworks.com
MathWorks®. (2017). patternnet. Retrieved on November 18 of 2017 from:
https://www.mathworks.com/help/nnet/ref/patternnet.html?requestedDomain=www.math
works.com
MathWorks®. (2017). rgb2gray. Retrieved on November 10 of 2017 from:
https://www.mathworks.com/help/matlab/ref/rgb2gray.html
135
Meditherm. (2017, July). DITI FAQ's. Retrieved from Meditherm. Digital Infrared Thermal
Imaging., on May 9 of 2017: http://www.meditherm.com/faq.htm
Mini_Physics©. (2017). The Electromagnetic Spectrum. Retrieved on November 11 of 2017
from: https://www.miniphysics.com/electromagnetic-spectrum_25.html
Mookiah, M. R. (2012). Data mining technique for breast cancer detection in thermograms using
hybrid feature extraction strategy. Quantitative InfraRed Thermography Journal, 9(2),
151-165.
Mora, L. A. (2017). Qué es la desviación estándar y cómo interpretarla. Retrieved on November
6 of 2017 from: https://tradingcenter.wordpress.com/2009/11/11/que-es-la-desviacion-
estandar-y-como-interpretarla-1/
Muñoz, M. (2016, january 21). CONACYT. Retrieved from CONACYT agencia informativa on
January 10 of 2017: http://conacytprensa.mx/index.php/ciencia/salud/5010-termografia-
para-la-deteccion-temprana-cancer-de-mama
NCI. (2016, December 7). National Cancer Institute. Retrieved from Mammograms, on January
10 of 2017: https://www.cancer.gov/types/breast/mammograms-fact-sheet#q1
NIH. (2017, May 9). Instituto Nacional del Cáncer. Retrieved from Aspectos generales de los
exámenes de detección del cáncer (PDQ®)–Versión para pacientes, on August 1 of 2017:
https://www.cancer.gov/espanol/cancer/deteccion/aspectos-generales-deteccion-paciente-
pdq
Nudelman, S. &. (1980). Chapter 12, Thermographic Instrumentation. In S. &. Nudelman,
Imaging for Medicine.
Parisky, Y. R., Sardi, A., Hamm, R., Hughes, K., Esserman, L., Rust, S., & & Callahan, K.
(2003). Efficacy of Computerized Infrared Imaging Analysis to Evaluate
Mammographically Suspicious Lesions. American Journal of Roentgenology, 263-269.
Pennes, H. H. (1948). Analysis of tissue and arterial blood temperatures in the resting human
forearm. Journal of applied physiology, 1(2), 93-122.
Puig, A. P. (1950). Curso teórico-práctico de ecuaciones diferencias aplicado a la Física y
Técnica. Biblioteca Matemática. Retrieved from Universidad del País Vasco. Campus
Gupuzkoa.300-303.
Radiansa_Consulting©. (2017). ¿Que son los campos eléctricos y magnéticos de baja
frecuencia? Retrieved on November 14 of 2017 from:
http://www.radiansa.com/contaminacion-electromagnetica/campos-
electromagneticos/campos-electricos-magneticos.htm
Rastghalam, R. &. (2016). Breast cancer detection using mrf-based probable texture feature and
decision-level fusion-based classification using hmm on thermography images. Pattern
Recognition, 51, 176-186.
Salazar, A. (2003). On thermal diffusivity. European journal of physics 24, no. 4, 351.
Salhab, M. A. (2005). The evolving role of the dynamic thermal analysis in the early detection of
breast cancer. International Seminars in Surgical Oncology, Vol. 2, No. 1, p. 8.
Sandham, J. (2005, 10 31). Medical Thermography. Retrieved from Ebme. Biomedical and
Clinical Engineering Website, on August 1 of 2017:
http://www.ebme.co.uk/articles/clinical-engineering/65-medical-thermography/
Scherer, D., Müller, A., & Behnke, S. (2010). Evaluation of Pooling Operations in Convolutional
Architectures for Object Recognition. Artificial Neural Networks – ICANN 2010., 92-
101.
136
Sherdeck, C. T. (2016). LBP y ULBP – Local Binary Patterns y Uniform Local Binary Patterns.
Retrieved on November 6 of 2017 from:
https://cesartroyasherdek.wordpress.com/2016/02/26/deteccion-de-objetos-vi/
Smith, J. O. (2007). 2. Complex Numbers. In Mathematics of the discrete Fourier transform
(DFT): with audio applications. Second Edition. (pp. 13-14). United States of America:
W3K Publishing.
Sossa Azuela, J. H., & Rodríguez Morales, R. (2011). Capítulo 2. Proceso de captación y
formación de una imagen. In Procesamiento y Análisis Digital de Imágenes (pp. 45-59).
Madrid: Alfaomega.
Sossa Azuela, J. H., & Rodríguez Morales, R. (2012). 3.3 Reducción del ruido en una imagen
por filtrado paso bajo. In Procesamiento y Análisis Digital de Imágenes. (pp. 105-106).
México, D. F.: Alfaomega.
TH-CEPREC. (2008-2009). Base de datos de termo-mastografías . Juchitán, Oaxaca, México.
Tuchin, V. V. (2007). 1. Optical Properties of Tissues with Strong. In Tissue optics: light
scattering methods and instruments for medical diagnosis. (pp. 91-93). Bellingham,
Washington USA: SPIE press.
Tyson, J. (2017). © How Stuff Works, Tech. Retrieved from How Night Vision Works on
November 22 of 2017 from: http://electronics.howstuffworks.com/gadgets/high-tech-
gadgets/nightvision2.htm
UNET. (2017). Fenómenos de transporte. Retrieved from Universidad Nacional Experimental
del Táchira on November 15 of 2017 from: http://www.unet.edu.ve/~fenomeno/F_DE_T-
165.htm
Vadivambal, R., & Digvir, S. J. (2016). Chapter 3. Classification, statistical and neural network.
In R. Vadivambal, & S. J. Digvir, Bio-Imaging. Principles, Techniques, and Applications.
(pp. 47-48). New York: Taylor & Francis Group.
Vega, J. P. (2017). Termoterapia. Retrieved from infoMED. Red de salud de Cuba. on
November 15 of 2017 from: http://www.sld.cu/galerias/pdf/sitios/rehabilitacion-
fis/termoterapia_1.pdf
Vo-Dinh, T. (2003). 17. Thermal Imaging for Biological and Medical Diagnostics. In Biomedical
Photonics (pp. 424-433). Florida: CRC Press.
Weisstein, E. W. (2017, 11 14). Temperature. Retrieved from Wolfram Research:
http://scienceworld.wolfram.com/physics/Temperature.html
Wille, D. (2011, 12 27). Caminar sobre brasas: “El poder de la mente sobre la materia”.
Retrieved from divulgadores.com on November 15 of 2017 from:
https://divulgadores.com/tag/capacidad-calorifica/
Zemansky, M. W. (1985). 1. Temperatura. In Calor y termodinámica (pp. 3-22). Madrid:
McGraw-Hill.
Zemansky, M. W. (1985). 17. Temas especiales. In Calor y termodinámica (pp. 482-488).
Madrid: McGraw-Hill.
Zemansky, M. W. (1985). 4. Calor y primer principio de la termodinámica. In Calor y
termodinámica (pp. 80-88). Madrid: McGraw-Hill.
