An
Independent Study Report
on
Enhancement of CVIP-FEPC and Color Normalization Program
Independent Study
(ECE 591)
SUBMITTED BY
Kumari Heema Poudel(800517296)
December 12, 2013
I
Abstract
Image analysis is an important part of Digital Image Processing which comprises of computer
vision and human vision applications. Usually, feature analysis and pattern classification are the
final and significant steps in image analysis. The Computer Vision and Image Processing Feature
Extraction and Pattern Classification Tool, CVIP-FEPC, was developed for exploring both
human and computer vision applications. The main function of this tool is to analyze feature
extraction and pattern classification process. This enables the user to perform batch processing
with large sets of image files which is much more efficient than processing one image at a time.
This tool allows the user to load multiple images and run experiments based on different
parameters by specifying the classes, selecting the features, selecting the test set, choosing the
pattern classification parameters and then letting the program to process on the entire image set.
Result files are stored in the output folder. One of the main objectives of this independent study
is to develop the next stable version of CVIP-FEPC that includes fixing the bugs present in
current CVIP-FEPC and to enhance the tool by adding more features.
Another main focus of this study is on the Color Normalization Program, which is used to do a
remapping to the color values of thermographic images so that all of the images are mapped to a
common temperature scale. Graphical User Interface (GUI) has been added to the Color
Normalization program to make it more user friendly and a mechanism has been developed to
organize the generated output files.
I
Acknowledgement
I would like to express my deepest appreciation to Dr. Scott E. Umbaugh for his continuous guidance and
encouragement throughout this independent study. Without his great supervision, constant help and
valuable suggestions this study would not have been fruitful.
Additionally I wish to thank Samrat Subedi and Jiyuan Fu; members of Computer Vision and Image
Processing Lab, Southern Illinois University Edwardsville for their help and support in testing the
application with the experiment on real images. I would like to thank Long Island Veterinary Specialists
Dr. Dominic J. Marino and Dr. Catherine A. Loughin for providing images which have been used in
testing the application. Last but not the least I want to thank Ravneet Kaur and Krishna Regmi for their
help.
Kumari Heema Poudel
II
Table of Contents
Abstract...................................................................................................................................................I
Acknowledgement.................................................................................................................................II
Table of Contents.................................................................................................................................III
List of Figures......................................................................................................................................IV
1. Introduction............................................................................................................................................1
2. Materials and Methods...........................................................................................................................2
2.1 Development Environment............................................................................................................2
2.2 Color Normalization......................................................................................................................2
2.3 CVIP-FEPC....................................................................................................................................4
3. Overall Flow of FEPC...........................................................................................................................9
4. Known Bugs.........................................................................................................................................10
5. Conclusion...........................................................................................................................................11
6. Future Work.........................................................................................................................................12
7. Appendices...........................................................................................................................................12
7.1 Color Normalization....................................................................................................................12
7.2 Screenshots..................................................................................................................................14
8. References............................................................................................................................................16
III
List of Figures
Figure 2.1: Color Normalization GUI and help file………………………………………………………...4
Figure 2.2: Exception occurred while using Leave One Out with Combinatoric test……………………...5
Figure 2.3: Help File of FEPC……………………………………………………………………………...6
Figure 2.4: Feature Selection screen with Variable Texture Distance……………………………………...7
Figure 4.1: Exception occurred while using the different feature category in second experiment……….11
Figure 7.1: Original Thermographic image and their normalized equivalents…………………………....14
Figure 7.2: Main window of CVIP-FEPC………………………………………………………………..14
Figure 7.3: Pattern Classification window of CVIP-FEPC ……………………………………………...15
Figure 7.4: Class List window of CVIP-FEPC …………………………………………………………15
IV
1. Introduction
This study is about the further development on Color Normalization program and CVIP–FEPC.
The CVIP–FEPC tool is extensively being used for the project ‘Veterinary Thermographic Image
Analysis’ from Long Island Veterinary Specialists, in which the feature analysis and pattern
classification is done for cats’ and dogs’ thermographic images to classify the cats and dogs for
normal and abnormal conditions of bone cancer, hyperthyroid and the gate analysis.
CVIP-FEPC is not limited for the analysis of thermographic images only. This can be used for
the analysis of any image types. One area of concern may be to run the experiments with
multiple texture distances at the same time. In previous version of CVIP-FEPC, this feature of
variable texture distance was not available. So, this is the one feature that has been added in the
CVIP-FEPC. A help file is always very useful for any application to make the user familiar with
the tool and this has been added in the current CVIP-FEPC.
Other than FEPC, work has been done on Color Normalization program which was developed
for the ‘Veterinary Thermographic Image Analysis’ to normalize the temperature range
according to the color of the thermographic image. A graphical user interface has been
developed to make the program user friendly and some other features have been added to
mitigate the manual work to manage the output files in folders. The new version of Color
Normalization program automatically creates the folders and keeps all the output files to their
respective folders.
This report will describe the changes made in FEPC to the program detail and their reflection in
GUI together with the changes in Color Normalization program.
1
2. Materials and Methods
2.1 Development Environment
CVIP-FEPC was developed with C#.NET programming language using Visual Studio 2008.The further
enhancements are done using the same programming language but with the Visual Studio 2010 and the
CVIP-FEPC is converted to C#.NET 2010. The following FEPC files are modified to achieve the variable
Texture Distance feature: MainForm.cs, FeatureHeader.cs, FeatureChooser.cs, FeatureVector.cs, and
ImageObject.cs.
Color Normalization program has been tested with the thermographic images provided by Long Island
Veterinary Specialists taken with a Meditherm Med2000 IRIS. The unit testing is done by the developer
whereas the functional testing is done by the researchers of the Veterinary Thermographic Image
Analysis.
Similarly for the FEPC features, unit and functional testing have been done with small set of images by
the developer, whereas researcher performed the functional testing on the real set of thermographic
images.
2.2 Color Normalization
The research project undergoing in SIUE, ‘Veterinary Thermographic Image Analysis’, uses
thermographic images of cats and dogs to classify them into different categories like normal and
abnormal. Within any thermographic image, each color represents a specific temperature. But in different
images, the same color may represent different temperatures, which can introduce noise in the pattern
classification process. To overcome this, a remapping is done to ensure that all the images are mapped to
a common temperature scale which is done in Color Normalization program. More about Color
Normalization methods can be found in the appendix [1].
2
2.2.1 Output Folder Structuring
In the previous version of Color Normalization, all the output normalized files were stored at the same
folder as input. It was a tedious task to manually manage the folders for placing normalized output files
and their masks for each category together. Now this version of the program makes this task easier by
automatically managing the organization of file.
Following folders are created automatically at the location where input files are located:
orig: All the input files are kept here together with the Mask folder.
lum: All lum normalized files are placed here concatenating ‘–lum’ in the original file name.
Similarly, mask file names are also changed by concatenating ‘-lum’ in mask file name and
these mask files are also copied together with the Mask folder within the Lum folder.
normGrey: All normGrey normalized files are kept here with concatenating ‘–normGrey’ in
the original file name. Similarly mask files name are also changed by concatenating ‘-
normGrey’ in mask file name and these mask files are also copied together with the Mask
folder within the normGrey folder.
normRGB: All normRGB normalized files are kept here with concatenating ‘–normRGB’ in
the original file name. Similarly mask files name are also changed by concatenating ‘-
normRGB’ in mask file name and these mask files are also copied together with the Mask
folder within the LnormRGB-lum folder.
normRGB-lum: All normRGB-lum normalized files will be kept here with concatenating ‘–
normRGB’ in the original file name. Similarly mask files name are also changed by
concatenating ‘-normRGB-lum’ in mask file name and these mask files are also copied
together with the Mask folder within the normRGB-lum folder.
2.2.2 GUI based Color Normalization
A simple GUI has been designed to make it user friendly. This GUI will load a *.txt file which contains
the path of the images followed by maximum and minimum temperatures of that image. The exact file
3
information is available in Help file of the application as shown in Figure 2.1.
Figure 2.1: Color Normalization GUI and help file
2.3 CVIP-FEPC
CVIP - FEPC is composed of four graphical interfaces: the Main window, the Feature Selection window,
the Pattern Classification Selection window and the Class List window. The main interface is the Main
window which is seen when the application is executed. The Feature Selection window appears when the
user clicks on Features button of Main window and Pattern Classification window appears when the
Classification button of the Main window is clicked. There is a setting menu in the main window which
directs to the Class List window.
The following works have been done in CVIP-FEPC:
2.3.1 Bug Fixing
The previous version of CVIP-FEPC was not working for Leave One Out method with Combinatoric test.
The following error was encountered when the user tried to execute CVIP-FEPC with the Leave one out
method and Combinatoric test, which is shown in Figure 2.2.
4
Figure 2.2: Exception occurred while using Leave One Out with Combinatoric test
After debugging, it was found that the training and test classes were being used without initializing them,
which was causing the error of “Object reference not set to an instance of an object”. For using any
method of a class, the class needs to be initialized and this was the missing part. After writing the codes
for initializing the class and calling its function in an appropriate manner, issue was fixed.
2.3.2 Help File
A help file has been added in the FEPC to make it easier to use and to have a better understanding of the
software. The help file is basically a Compiled HTML file (CHM file) which is designed and compiled
using ‘HTML Help Workshop’ software. Snapshot for the help file is shown in Figure 2.3 below.
5
Figure 2.3: Help File of FEPC
2.3.3 Variable Texture Distance:
In the current version of FEPC, we have the option of selecting one texture distance at a time. This means
any experiment even if it is combinatoric, it will run with only one texture distance. What if a user wants
to analyze the output of experiment with several texture distances with the same feature set? For this, the
user has to run the experiment several times by passing the texture distance of choice. A better option is
to incorporate the choice of selecting multiple texture distances in FEPC itself.
With the option of selecting variable texture distance, the feature selection window looks like as shown in
Figure 2.4.
6
Figure 2.4: Feature Selection screen with Variable Texture Distance
This variable texture distance feature is available only if the experiment is to be run for ‘Combinatoric’
option and not for the ‘Single Test’. There is a button Switch to Single Test Mode, which can be selected
if the user wants to run Single Test experiment only.
With the addition of this new feature the first file that gets affected is MainForm.cs. In this file the
program checks whether the user has selected the variable texture option or not. If the multiple texture
distance is selected, the program calculates the different texture distances (d1, d2,…dn) and the
experiment runs with all the texture distances.
7
The second file that gets affected is FeatureDlg.cs. This is the interface for feature selection and three
boxes have been added here which are the parameters for variable texture distance:
i. Lower: This is the first texture distance value with which the experiment will start its
execution.
ii. Increment: This is the increment on the lower texture distance.
iii. Upper: This is the final limit of the texture distance.
To illustrate this process, let us take an example where lower is set as 2, increment is set as 2 and upper is
set as 8.
Then formula used to find the iteration is
N= ((Upper –Lower)/Increment) + 1
D[1]=lower;
D [ n ]=D [ n−1 ]+increment
Where n varies from 2 to N
So in our example N= ((8-2)/2)+1=4
So d1=lower=2
D2=d1+increment=2+2=4
D3=d2+increment =4+2=6
D4=d3+increment=6+2=8
So the experiment will run with these 4 texture distances along with all other features and classification
algorithms selected at the beginning.
The output files of the experiment will be saved into four folders, with four different texture distances.
There can be cases where the user wants to run the combinatoric experiments but want to select just one
texture distance. For this, the desired texture distance need to be typed in lower box and upper box should
have the same value. Value entered in the increment box does not matter here.
FeatureChooser.cs and FeatureHeader.cs are the files where output files are being written. So,
8
appropriate texture distance is being passed to be written in the output files in case of multiple texture
distances.
FeatureVector.cs and ImageObject.cs are the files which calculate the feature vectors, so proper distance
is being passed while calculating the texture features for the variable texture distance case.
3. Overall Flow of FEPC
The Main Window (MainForm.cs) appears when a user runs the application. In this window, the user can
add multiple images at the same time. Now the user can select the features by clicking on Features button
and this is the point where program control goes over to FeatureDlg.cs which is the interface for feature
selection window. All the binary, histogram, RST invariant, texture and spectral features are listed here
which can be selected.
After selecting the features user can set the parameters: data normalization methods, distance and
similarity measures and classification algorithm for pattern classification. If combinatoric test has been
selected, the user can choose multiple methods for every category otherwise the user can select only one
method for each category. After selecting the classification methods, the user will classify the images to
any classes like normal and abnormal and hit the button Run Test.
MainForm.cs is the file where all the set up for running the experiment gets done. This file handles the
deleting of images (response of Delete button in the same window) and changing the output location
(action of clicking on Browse button). FeatureVector.cs is the file which calculates all the feature vectors.
Normalizer.cs is the file where all the calculations regarding data normalizations are done.
FeatureHeader.cs is the file which generates all the text output feature files. FeatureFile.cs is used for
getting the feature files which are to be written in the output directory. This file checks if the user has
selected any of the normalization method, if yes this file directs the program control to do the
normalization and then get the feature file otherwise will get the feature file without any normalization
9
being done. FeatureChooser.cs file checks for the features which are selected by the users and calculates
the number of combinations with the number of features selected. For any combinatoric test, the total
number of combinations will be: 2n, where n is the total number of the features and pattern classification
methods selected for each texture distance. The Classifier.cs file checks for the classification methods
being selected by the user and recalculates the number of combinations since the number of features is
increased with the selection of classification methods and thus prepares the content for the output file.
4. Known Bugs
A bug was encountered in the previous version of CVIP-FEPC. If the user runs the application for the
first time and has selected a few categories of the features, then the application will work perfectly, but if
the user tries to run another experiment without restarting the application and selects the features of the
categories not selected previously then the error will pop up.
For example, if the user has selected features from binary features, histogram and texture features
categories in the first experiment and runs the experiment on these feature set, the experiment runs
successfully. But now with the addition of any feature from the RST-invariant category into the feature
set and running the experiment will produce an error. The error message is shown in the snapshot in
Figure 4.1 below. This was noticed at the later stage of implementing the variable texture distance, so this
still exists in the current application.
10
Figure 4.1: Exception occurred while using the different feature category in second experiment
5. Conclusion
All the bugs except for the one mentioned in the Known Bugs section are successfully fixed. The major
concern was the bug which was preventing the use of Leave One Out method with Combinatoric Test,
which is very important functionality of CVIP-FEPC, is now fixed. The newly added help file includes
information about overview of CVIP-FEPC and how to use it. The variable texture distance feature is also
working successfully for both Training and Test Set and Leave One Out, except of the bug mentioned
above.
Enhancements of Color Normalization program are also giving the expected results. Output files are
structured and stored in a consistent way and the help file includes the necessary information regarding
11
how to create the *.txt file to feed as input in the same program.
6. Future Work
In the near future, the first task would be to fix all the known bugs so that the application will be bug free.
The other work areas are in the ring and the sector option of feature selection page where the user will be
able to select multiple values of rings and sectors at the same time. The main purpose of all these is to
make all the features able to have dynamic values which are static for now so that the user will have the
ability to run the Combinatoric test with all the choices of features in a single experiment.
7. Appendices
7.1 Color Normalization
Four different Color Normalization methods are used:
Luminance: For each pixel in the original image, the color components are weighted as follows
to generate a gray level value: Color components are calculated by the following formula to
generate a grey level value for each pixel of image:
GrayLevel=0 .3∗Red+0 .6∗Green+0 .1∗Blue
Norm-Gray: In this method, the temperature values between 15° C and 45° C are mapped to gray
levels from 0 to 255. For this, the temperature represented by the color of the corresponding pixel
in the original image is used to map each pixel in the normalized image by coloring with the gray
level for that temperature. Then, a histogram stretch is done as the final step of this normalization
method that is obtained by saving the image as a TIFF file.
Norm-RGB: This process is similar to the Norm-Gray but temperatures are mapped to a
continuous version of the original color palette.
12
Norm-RGB-Lum: This method is done in two step process: first Norm-RGB is applied to
generate gray scale image and then luminance normalization is applied to the resultant image [1].
The original thermographic images for the research have been provided by the Long Island Veterinary
Specialists.
Figure 7.1(a): Original Thermographic Image Figure 7.1(b): Luminance Normalized Image
Figure 7.1(c): Norm Grey Normalized Image Figure 7.1(d): Norm-RGB Normalized Image
13
Figure 7.1(e): Norm-RGB-Lum Normalized Image
Figure 7.1: Original Thermographic image and their normalized equivalents
7.2 Screenshots
The following are the screenshots of different windows seen during execution of CVIP-FEPC.
Figure 7.2: Main window of CVIP-FEPC
14
Figure 7.3: Pattern Classification window of CVIP-FEPC
Figure 7.4: Class List window of CVIP-FEPC
15
8. References
[1] Umbaugh, E S.,Solt P. (2008), “Veterinary Thermographic Image Analysis Data and
Temperature Normalization”.
[2] CVIPtools(2010), “Computer Vision and Image processing Tools”.
http://cviptools.ece.siue.edu
[3] Umbaugh, E S. (2010), “Digital Image Processing and Analysis: Human and Computer
Vision Applications with CVIPtools, Second Edition”. The CRC Press, Boca Raton, FL, 2010.
16