Neural Networks Apps (v. Short, For Newbies)

8/21/2019 Neural Networks Apps (v. Short, For Newbies)

1/16

Kokkoras F. | Paraskevopoulos K. P a g e | 1

E x e r c i s e Artificial Neural Networks

ANNs

In this Exercise

1.

Introduction

2.

ANNs and Matlab

3.

Part #1 (Fitting a Function / Regression)4. Part #2 (Pattern Recognition / Classification)

5. Part #3 (Clustering)

6.

References

Duration: 120 min

1. Introduction (10 min)

An Artificial Neural Network (ANN from now on) is a mathematical structure which consists of inter-

connected artificial neurons that mimics, in a much smaller scale, the way a biological neural network

(or brain) works. An ANN has the ability to learn from data, either in a supervised or an unsupervisedfashion and can be used in tasks such as regression, classification, clustering and more.

A typical artificial neuron is depicted in Figure 1. The scalar inputs pi are transmitted through con-

nections that multiply their strength by the scalar weight w i to form the product wipi, again a scalar.

All the weighted inputs wipi are added and to Σwipi we also add the scalar bias b. The result is the ar-

gument of the transfer function f , which produces the output a. The bias is much like a weight, ex-

cept that it has a constant input of 1.

Figure 1: An artificial neuron with one input and bias.

Note that wi and bi are both adjustable scalar parameters of the neuron. The central idea of neu-

ral networks is that such parameters can be adjusted so that the network exhibits some desired or

interesting behavior. Thus, you can train the network to do a particular job by adjusting the weight or

bias parameters, or perhaps the network itself will adjust these parameters to achieve some desiredend. The most commonly used transfer functions are the hard-limit (or step), the linear and the sig-

moid (or logistic), all depicted in Figure 2.

Figure 2: Commonly used transfer functions: hard-limit (left), linear (middle) and sigmoid (right).

The most common ANN architecture consists of many neurons organized in layers. Each neuron in

a layer is connected to all the neurons of the next layer. We can distinguish between input, hidden

and output layers. There is only one input and output layer whereas more than one hidden layers are

Σ b

1

p1 w1

fa

a=f( Σ w i pi +b)

p2 w2

pn

wn pi

wi


2/16

International Hellenic University Artificial Neural Networks


allowed. Note also that it is common for the number of inputs to a layer to be different from the

number of neurons. Some other definitions you should aware of are:

learning rules: procedures for modifying the weights and biases of a network, that is, meth-

ods of deriving the next changes that might be made in an ANN

training: a procedure whereby a network is actually adjusted to do a particular jobThere are two broad categories of learning rules. In supervised learning the network is provided

with a set of examples (the training set) of proper network behavior (pairs of input and known/cor-

rect output (target)). As the inputs are applied to the network, the calculated network outputs are

compared to the targets. The learning rule is then used to adjust the weights and biases of the net-

work in order to move the network outputs closer to the targets. In unsupervised learning, the

weights and biases are modified in response to network inputs only. There are no target outputs

available.

For the purpose of training, the input data are divided into three sets:

training: it is used for adjusting the weights and biases validation: it is used to decide when to stop the training process, to avoid overfitting, a situa-

tion where the network memorizes the training data, rather than learning the law that gov-

erns them

testing: it is used to measure the performance of the trained network – it is important that

this data do not participate in the training process

The most common forms of ANNs are the multilayered perceptrons, the self-organized maps

(SOM or Kohonen Networks) and the associative memories (also known as Hopfield networks).

3. ANNs and Matlab (5 min) Matlab (R2008a -v.7.6- was used for the tutorial) provides the Neural Networks Toolbox , a set of

tools that include GUIs, wizards and functions that allow any user level (from novice to expert) to use

and experiment with neural networks with minimal effort. The easiest way to use the toolbox is

through the GUIs that perform certain tasks. In this tutorial we will see the following tasks:

fit a function / regression

recognize patterns / classification

cluster data

The second easiest way to use the toolbox is through basic command-line operations. The com-

mand-line operations offer more flexibility than the GUIs, but with some added complexity. In this

tutorial we will use both GUIs and command line operations.

In the following exercises we will see all the steps pertained to the ANN usage to solve regression,

classification and clustering problems. These steps include: collecting data, creating the network,

configuring the network, initializing the weights and biases, network training, network validation and

network use. Note though that you cannot use any type of ANN for any type of problem.

The datasets used in the following were taken from the Bren School of Information and Computer

Science at the University of California, Irvine (Repository Of Machine Learning Databases) [1]. They

are available at:

ftp://ftp.ics.uci.edu/pub/machine-learning-databases/

The following exercises are adaptations of Matlab's tutorials to local needs.

ftp://ftp.ics.uci.edu/pub/machine-learning-databases/ftp://ftp.ics.uci.edu/pub/machine-learning-databases/ftp://ftp.ics.uci.edu/pub/machine-learning-databases/


3/16



4. Part #1 (Fitting a Function / Regression) (35 min)

Neural networks are good at fitting functions. In fact, there is proof that a fairly simple neural net-

work can fit any practical function.

Problem: Using data from a housing application [1] we want to design a network that can predict the

value of a house (in $1000s), given 13 pieces of geographical and real estate information. The dataset

consists of 506 example homes (records) for which we have 13 items of data and their associated

market values. The dataset is available at:

ftp://ftp.ics.uci.edu/pub/machine-learning-databases/housing/

The file housing.data contains the data whereas housing.names describes what this data is about.

Take your time to familiarize yourself with the dataset. The better you know your data the more in-

sight can give you during ANN modeling.

Defining a Regression Problem

Data used for regression problems in Neural Network Toolbox, should be arranged in columns (that

is, a data record or training instance is a column). Additionally, input and target values should be lo-

cated in separate matrixes or files. For example, assuming data for modeling the logical AND prob-

lem, one should provide:

inputs = [0 1 0 1; 0 0 1 1];

targets = [0 0 0 1];

Alternatively, two delimited text files containing:

0 1 0 1

0 0 1 1and:

0 0 0 1

respectively, are adequate. This might be a little strange because we usually arrange records in sepa-

rate lines, that is:

0 0 0

0 1 0

1 0 0

1 1 1

Using command line functions

1. To load the data in Matlab type (in Matlab's command line):

load houseInputs.txt

load houseTargets.txt

These two data files where created from housing.data for your convenience. Since this dataset is

also available in Matlab, you can alternatively just type:

load house_dataset

2. Create a network. For this example, you use a feed-forward network with the default tan-sigmoid

transfer function in the hidden layer and linear transfer function in the output layer. This struc-ture is useful for function approximation (or regression) problems. Use 20 neurons (somewhat ar-

ftp://ftp.ics.uci.edu/pub/machine-learning-databases/housing/ftp://ftp.ics.uci.edu/pub/machine-learning-databases/housing/ftp://ftp.ics.uci.edu/pub/machine-learning-databases/housing/


4/16



bitrary) in one hidden layer. The network has one output neuron, because there is only one target

value associated with each input vector.

net = newfit(houseInputs,houseTargets,20);

More neurons require more computation, but they allow the network to solve more compli-

cated problems. More layers require more computation, but their use might result in the network

solving complex problems more efficiently. Beware though that adding too many neurons com-

pared to the dataset size might result in overfitting phenomena (more on this later).

3. Train the network. The network uses the default Levenberg-Marquardt algorithm for training

(backpropagation-like). The application randomly divides input vectors and target vectors into

three sets as follows:

60% are used for training,

20% are used to validate that the network is generalizing and to stop training before overfitting.

t 20% are used as a completely independent test of network generalization.

To train the network, enter:

net=train(net,houseInputs,houseTargets)

During training, the training window (right) opens.

This window displays training progress and allows

you to interrupt training at any point by clicking Stop

Training.

The train function presents all the input vectors to

the network at once in a batch. Alternatively, you can

present the input vectors one at a time using theadapt function.

As you can see in the screenshot, the training

stopped when the validation error increased for six

iterations, which occurred at iteration (or epoch) 13.

Note: Given the random initialization of

the network, every 'run' produces different

results. You will get different results from

these depicted here but if the modeling pro-

cess goes well, you should expect results of

the same quality.

If you click Performance in the training

window, a plot of the training errors, valida-

tion errors, and test errors appears, as shown

in the figure on the left. In this example, the result is reasonable because of the following considera-

tions:


5/16



The final mean-square error is small (2.6).

Both the test set error and the validation set error have similar characteristics (green and red

lines in the plot).

No significant overfitting has occurred by iteration 7 (where the best validation performance

occurs).

4. Perform some analysis of the network response. If you click Regression in the training window,

you can perform a linear regression between the network outputs and the corresponding targets.

The following figure shows the results. Ideally (with zero error), the points should be placed on

the target=output line. In our case, the output tracks the targets very well for training, testing,

and validation, and the R-value is a bit over 0.95 for the total response.

If you need more accurate results, you

can try any of these approaches:

Reset the initial network weights and

biases to new values with newfit andtrain again.

Increase the number of hidden neurons.

Increase the number of training vectors

(more data).

Increase the number of input values, if

more relevant information is available.

Try a different training algorithm.

In this case, the network response is satis-

factory, so we can use the ANN to predict

the value of a house, given the 13 input parameters.

To get a better insight on how the training proceeds, keep the Performance and Regression plots

open and initialize and re-train the network while watching the plots animate.

Improving Results

This example demonstrated some simple commands you can use to solve many types of problems.

However, if your first attempt does not meet your needs or expectations do not hesitate to retry. If

the network is not sufficiently accurate, you can try initializing the network and the training again.

Each time your initialize a feed-forward network, the network parameters are different and might

produce different solutions.

To re-initialize the network type: net = init(net); where net is your neural network.

To re-train the network type: net = train(net,houseInputs,houseTargets);

As a second approach, you can increase the number of hidden neurons above 20. Larger numbers

of neurons in the hidden layer give the network more flexibility because the network has more pa-

rameters it can optimize. You should increase the layer size gradually. If you make the hidden layer

too large, you might cause the problem to be under-characterized and the network must optimize

more parameters than there are data vectors to constrain these parameters. Your network will beoverfitted. It will memorize the training examples and although it will demonstrate high accuracy on


6/16



the training data, it will work poorly on never-seen data. A better approach is to try using additional

training data. Providing additional data for the network is more likely to produce a network that gen-

eralizes well to new data.

Using the Neural Network Toolbox Fitting Tool GUI

1. Open the Neural Network Toolbox Fitting Tool with the command: nftool

2. Click Next to proceed.

3. Click the first button to load houseInputs.txt

as input parameters and the second one to load

the houseTargets.txt as target parameters. You

should browse for your files in the Import Wizard

and check that the data are properly recognized

(you will have no issue with the provided data

files – just hit the 'Next' button twice and then Finish). Alternatively, you can load data alreadyloaded in the workspace. Click the Next button.

4. Click 'Next' to display the Validate and Test Data

window, shown in the figure on the right. Here

you can select the portion of the original dataset

that will be used for validation and testing. Keep

the default values of 15%.

5. Click Next. The number of hidden neurons is set

to 20. You can change this value in another run if

you want. You might want to change this number if the network does not perform as well as youexpect.

6. Click 'Next' to see a synopsis of the situation and then click on the button. From now on

the situation is known since you

get the same windows as in the

command line approach done

earlier. As mentioned, the re-

sults are not exactly the same

but are very similar. The fit is

almost perfect for training, test-ing, and validation data.

7. Click 'Next' in the Neural Net-

work Toolbox Fitting Tool (win-

dow) to evaluate the network.

At this point, you can test the

network against new data (us-

ing the controls on top-right).

If you are dissatisfied with the network's performance on the original or new data, you can take

any of the following steps (by using the buttons on the left):


7/16



Train it again.

Increase the number of neurons.

Get a larger training data set.

8. If you are satisfied with the network performance, click Next to go to the Save Results window.

You can use the buttons on this screen to save your results. You can also click Generate M-File tocreate an M-file that can be used to reproduce all of the previous steps from the command line.

Creating an M-file can be helpful if you want to learn how to use the command-line functionality

of the toolbox to customize the training process.

9. When you have saved your results, click Finish.

5. Part #2 (Pattern Recognition / Classification) (35 min)

Neural networks are also good at recognizing patterns. In this exercise we will classify a tumor as be-

nign or malignant, based on uniformity of cell size, clump thickness, mitosis, etc. [3]. Out dataset

consists of 699 example cases for which we have 9 items of data and the correct classification as be-

nign or malignant. This breast cancer databases was originally obtained from the University of Wis-

consin Hospitals, Madison from Dr. William H. Wolberg. It is available online at:

ftp://ftp.ics.uci.edu/pub/machine-learning-databases/breast-cancer-wisconsin/

Note that this dataset contains some missing values (the numbers are replaced with question

mark characters '?'). For the needs of this tutorial, we have replaced the question marks with the

number 3.5. The dataset is also available in Matlab but except from the missing values issue, all

numbers are divided by 10. Finally, the class designators are 2 and 4 in the original dataset but 0 and

1 in the Matlab's dataset. We will also use these designators (0 and 1). More on this right in the fol-

lowing text.

Defining a Classification Problem

As with the function fitting problem discussed in the previous section, records should be organized in

columns (vectors) in a matrix (or file). Additionally, the same count of target vectors are required to

indicate the classes to which the input vectors are assigned. There are two approaches to creating

the target vectors:

If there are only two classes we set each scalar target value to either 1 or 0, indicating which

class the corresponding input belongs to.

For example, you can define the exclusive-or classification problem as follows:

inputs = [0 1 0 1; 0 0 1 1]

targets = [0 1 1 0];

If inputs are to be classified into N different classes then we create target vectors (again ar-

ranged as columns) that consists of N elements, where for each target vector, one element is 1

and the others are 0.

For example, in the exclusive-or problem, we can write the target vectors like:

targets = [1 0 0 1; 0 1 1 0];

That is, classification problems involving only two classes can be represented using either for-

mat. The targets can consist of either scalar 1/0 elements or two-element vectors, with one el-ement being 1 and the other element being 0.

ftp://ftp.ics.uci.edu/pub/machine-learning-databases/breast-cancer-wisconsin/ftp://ftp.ics.uci.edu/pub/machine-learning-databases/breast-cancer-wisconsin/ftp://ftp.ics.uci.edu/pub/machine-learning-databases/breast-cancer-wisconsin/


8/16



For convenience, we write this dataset in the way it will be written in a text file. Inputs will be:

0 1 0 1

0 0 1 1

whereas target classes will either be:

0 1 1 0 (direct indication of the class)

or 1 0 0 1 (1st class – we put 1 if this is our case, 0 otherwise)

0 1 1 0 (2nd class – we put 1 if this is our case, 0 otherwise)

The next section demonstrates how to train a network from the command line, after you have

defined the problem.

Using Command-Line Functions

1. Load the tumor classification data as follows:

load cancerInput.txt

load cancerTarget.txt

2. Create a network. For this example, you use a pattern recognition network, which is a feed-forward network with tan-sigmoid transfer functions in both the hidden layer and the output lay-

er. As in the function-fitting example, we will use 20 neurons in one hidden layer.

The network has two output neurons, because there are two categories associated with each

input vector. Each output neuron represents a category. When an input vector of the appropriate

category is applied to the network, the corresponding neuron should produce a 1 and the other

neurons should output a 0.

To create a network, enter this command:

net = newpr(cancerInputs,cancerTargets,20);

3. Train the network. The pattern recognition network

uses the default Scaled Conjugate Gradient algorithm

for training. The application randomly divides the input

vectors and target vectors into three sets:

60% are used for training.

20% are used to validate that the network is gener-

alizing and to stop training before overfitting

The last 20% are used as a completely independent

test of network generalization.

To train the network, enter this command:

net=train(net,cancerInputs,cancerTargets);

During training, as in function fitting, the training

window opens (see screenshot on the right). This win-

dow displays training progress. To interrupt a long train-

ing at any point, click 'Stop Training'.

This example uses the train function. It presents all


9/16



the input vectors to the network at once in a batch. Alternatively, you can present the input vec-

tors one at a time using the adapt function. See Matlab's Help System for more on this.

This training stopped when the validation error increased for 6 iterations, which occurred at

epoch 21.

4. To find the validation error, click 'Performance' in the training window. A plot of the training er-

rors, validation errors and test errors appears, as shown in the figure on the right. The best valida-

tion performance occurred at iteration 15 and the network at this iteration (epoch) is returned.

5. To analyze the network response, click 'Confusion'

in the training window. A display of the confusion

matrix appears (figure on the right) that shows vari-

ous types of errors that occurred for the final trained

network.

There are 4 tables: each one displays the network

response for the training, validation, testing and all

datasets. The diagonal (green) cells in each table

show the number of cases that were correctly classi-

fied, and the off-diagonal cells (red) show the mis-

classified cases. The blue cell in the bottom right

shows the total percent of correctly classified cases

(in green text) and the total percent of misclassified

cases (in red text). The results for all three data sets (training, validation and testing) show very

good recognition (that is, the network response is satisfactory). If you needed even more accurate

results, you could try any of the following approaches:

Reset the initial network weights and biases to new values with init (re-build) and train again.

Increase the number of hidden neurons.

Increase the number of training vectors.

Increase the number of input values, if more relevant information is available.

Try a different training algorithm.

To get more experience in command-line operations you can open a plot window (such as the

performance plot) during training and watch it animate.

Using the Neural Network Toolbox Pattern Recognition Tool GUI1. Open the Neural Network Toolbox Pattern Recognition Tool window with the command: nprtool

2. Click 'Next' to proceed. The Select Data

window opens (see figure on the right). If

you have the datasets (input and target) al-

ready loaded in the workspace (as is the

case of the figure), then you can select

them from the proper combo boxes. Oth-

erwise you can click the button to start

the Import Wizard where you browse forthe proper file. There exist additional da-


10/16



tasets accessible from the 'Load Example Da-

taset' button.

3. Click 'Next' to continue to the Validate and

Test Data window, shown in the figure on the

right. Validation and test data sets are eachset to 15% of the original data. You can keep

these values as this is the usual case.

4. Click 'Next' . The number of hidden neurons is

set to 20. You can change this in another run if you want. You might want to change this number

if the network does not perform as well as you

expect.

5. Click 'Next' to go to the Train Network window

where a synopsis of the parameters set so far is

displayed.

6. Click the 'Train' button. The training continued for

32 iterations and stopped when the validation er-

ror increased for 6 iterations, which occurred at

epoch 32-6=26.

7. In the NN Training window Under the Plots pane,

click Performance to see how the training phase

proceeded.

We remind here that due to random ini-

tialization of weights and biases in the artifi-

cial neurons, you cannot get exactly the same

results as depicted here.

8. In the NN Training window Under the Plots pane,

click Confusion. The figure displayed (right) shows

the confusion matrices for training, testing and

validation, as well as the three kinds of data com-

bined. The network's outputs are almost perfect,

as you can see by the high numbers of correct re-

sponses in the green squares and the low numbers

of incorrect responses in the red squares. The


11/16



lower right blue squares illustrate the overall accuracies. In these squares, green text is used for

the correct responses and red for the incorrect.

9. In the Neural Network Toolbox Pat-

tern Recognition Tool under the Plots

pane, click 'Receiver Operating Char-acteristic' (ROC). The colored lines

(green and blue) in each axis repre-

sent the ROC curves for each of the

two categories of this problem. The

ROC curve is another visualization of

the quality of our network. It is a plot

of the true positive rate (sensitivity)

versus the false positive rate (1 -

specificity) as the threshold is varied.

A perfect test would show points in

the upper-left corner, with 100%

sensitivity and 100% specificity. For

this simple problem, the network performs almost perfectly.

10. In the Neural Network Toolbox Pattern Recognition Tool, click 'Next' to evaluate the network. At

this point, you can test the network against new data. If you are dissatisfied with the network's

performance on the original or new data, you can train it again, increase the number of neurons,

or perhaps get a larger training data set.

11. When you are satisfied with the network performance, click 'Next' . Use the buttons on this screen

to save your results in the workspace. If you click Generate M-File, the tool creates an M-file, with

commands that recreate the steps that you have just performed from the command line. Gener-

ating an M-file is a good way to learn how to use the command-line operations of the Neural

Network Toolbox software.



12/16



6. Part #3 (Clustering) (35 min)

Clustering data is another excellent application for neural networks. This process involves grouping

data by similarity. For example, you might perform:

Market segmentation by grouping people according to their buying patterns

Data mining by partitioning data into related subsets

Bioinformatic analysis by grouping genes with related expression patterns

In this exercise we will cluster flower types according to petal length, petal width, sepal length,

and sepal width. We will use the famous 'IRIS' dataset [4] which consists of 150 example cases for

which we have these four measurements. You can get the dataset from the following URL:

ftp://ftp.ics.uci.edu/pub/machine-learning-databases/iris/

Take your time and examine the description of the dataset (file iris.names). Getting familiar with

your dataset usually gives you insight on how to proceed.

Defining a Clustering Problem

To define a clustering problem, simply arrange the input records (vectors) to be clustered, as col-

umns in an input matrix. For instance, you might want to cluster this set of 10 two-element vectors:

inputs = [7 0 6 2 6 5 6 1 0 1; 6 2 5 0 7 5 5 1 2 2]

The next section demonstrates how to train a network from the command line, after you have

defined the problem. You will use the irisInput.txt file which is the initial dataset iris.data dataset

with the class attribute removed and the rest of the array transposed1 to meet the Matlab require-

ment of having the data records in columns (they are arranged as rows in the iris.data file).

Using Command-Line Functions

1. Load the data with the command: load irisInputs.txt

Note that the initial dataset (iris.data) consists of input

vectors and target vectors. However, you only need the

input vectors for clustering. This is because, unlike the

previous examples, here we are going to use unsuper-

vised learning.

2. Create a network. For this example, you use a self-orga-

nizing map (SOM). This network has one layer, with the

neurons organized in a grid. When creating the network,

you specify the number of rows and columns in the grid:

net = newsom(irisInputs,[6,6]);

3. Train the network. The SOM network uses the default

batch SOM algorithm for training.

net=train(net,irisInputs);

1

It's easy to transpose an array (turn rows in to columns). Open the dataset in MS Excel, Copy the array and Paste Specialselecting Transpose. You can do the transposition in Matlab as well but providing the exact steps is beyond the scope of this

section.

ftp://ftp.ics.uci.edu/pub/machine-learning-databases/iris/ftp://ftp.ics.uci.edu/pub/machine-learning-databases/iris/ftp://ftp.ics.uci.edu/pub/machine-learning-databases/iris/


13/16



4. During training, the training window opens (see figure on the right) and displays the training pro-

gress. To interrupt training at any point, click the 'Stop Training' button.

5. For SOM training, the weight vector associated with each neuron moves to become the center of

a cluster of input vectors. In addition, neurons that are adjacent to each other in the topology

should also move close to each other in the input space. The default topology is hexagonal. Toview it, click 'SOM Topology' from the network training window.

In this figure (left), each of the hexagons represents a neu-

ron. The grid is 6-by-6, so there are a total of 36 neurons in this

network. There are four elements in each input vector, so the

input space is four-dimensional. The weight vectors (cluster cen-

ters) fall within this space.

Because this SOM has a two-dimensional topology, you can

visualize in two dimensions the relationships among the four-

dimensional cluster centers. One visualization tool for the SOMis the weight distance matrix (also called the U-matrix).

6. To view the U-matrix, click 'SOM Neighbor Dis-

tances' in the training window. In this figure

(right), the blue/purple hexagons represent

the neurons. The red lines connect neighboring

neurons. The colors in the regions containing

the red lines indicate the distances between

neurons. The darker colors represent larger

distances, and the lighter colors representsmaller distances.

A band of dark segments crosses from the

lower-center region to the upper-right region.

The SOM network appears to have clustered

the flowers into two distinct groups, each one

located on one side of this dark band.

7. Now click the 'SOM Sample Hits' button to see

another visualization (see figure on the right).

The plot displayed calculates the classes foreach flower and shows the number of flowers

in each class. Areas of neurons with large

numbers of hits indicate classes representing

similar highly populated regions of the feature

space. Areas with few hits indicate sparsely

populated regions of the feature space. Sum-

ming up all numbers results to 150. (Why?)

As mentioned in earlier parts of this tutorial, during training, you can have a plot window open

(such as the SOM weight position plot) and watching it animate as the training proceeds.


14/16



Note: If you have past experience with the IRIS dataset, you will probably know that there exist

three classes: Iris-Setosa, Iris-Versicolor

and Iris-Virginica (you can verify this by

looking into the dataset). You probably

wonder why our clustering network

identified only two. To better under-

stand the situation (and the value of fa-

miliarize yourself with the data), check

some visualization of them in 2D (figure

on the left) using each time 2 of the 4

available dimensions.

As you can see, only one of the clas-

ses is linearly separable from the others

(the one displayed with blue points). The

other two are not separable, so our ANN failed to identify them and considered them as a single class(cluster).

Unlike clustering (which fails to identify the three clusters in the IRIS data), you can train a highly

accurate (>97%) classification ANN if you work with the IRIS data like in Part#2 of this tutorial. You

will also need target data since this will be a supervised training. We have them prepared already for

your convenience in the file irisTargets_x.dat . Try it yourself as an exercise.

The situation discussed here does not, in any case, reduce the usefulness of the capability of

ANNs to identify clusters in unknown datasets. For this particular dataset, you can find other tech-

niques in literature (for example, fuzzy clustering) that produce better clustering results.

Using the Neural Network Toolbox Clustering Tool GUI

1. Open the Neural Network Toolbox Clustering Tool window with this command: nctool

2. Click Next. The Select Data window appears. This time we will use a different iris dataset existing

in the toolbox. Click in the button, select 'Simple Clusters' and press the

button. In this dataset two-element vectors are assigned to four classes. As mentioned at the end

of the previous section, samples may be classified using clustering (using only input data) or with

pattern recognition (classification) or even fitting (with input and target data).

The dataset consists of 1000 records. 'simpleclusterInputs' is a 2x1000 matrix of values. 'sim-

pleclusterTargets' is an 4x1000 matrix, where each ith column indicates which category the ith iris

belongs to, with a 1 in one element and zeros in the other elements (see figure below).

This simple example can be solved with the Neural Network Pattern Recognition Tool (nprtool )

or Clustering Tool (nctool ). Here we follow the later approach. You can try the former as an exercise.


15/16



3. In the Select Data window click 'Next' to continue to the Network Size window. The size of the

two-dimensional map is set to 10. This map represents one side of a two-dimensional grid. The to-

tal number of neurons is 100. You can change this number in another run if you want.

4. Click Next . The Train Network window appears.

5. Click Train. The training runs for the maxi-

mum number of epochs, which is 200.

6. Investigate some of the visualization tools

for the SOM. Under the Plots pane, click

SOM Sample Hits. This figure (right) shows

how many of the training data are associat-

ed with each of the neurons (cluster cen-

ters). The topology is a 10-by-10 grid, so

there are 100 neurons. The maximum num-

ber of hits associated with any neuron is 32.Thus, there are 31 input vectors in that clus-

ter. We have marked the four classes with

red circles. As you ca see, the two classes on

the right are not well-separable.

7. You can also visualize the SOM by displaying weight places (also referred to as component

planes). Click SOM Weight Planes in the Neural Network Toolbox Clustering Tool. This figure

shows a weight plane for each element of

the input vector (two, in this case). They are

visualizations of the weights that connecteach input to each of the neurons. (Darker

colors represent larger weights.) If the con-

nection patterns of two inputs were very

similar, you can assume that the inputs are

highly correlated. In this case, input 1 has

connections that are very different than

those of input 2.

8. In the Neural Network Toolbox Clustering Tool, click Next to evaluate the network. At this point

you can test the network against new data. If you are dissatisfied with the network's performance

on the original or new data, you can increase the number of neurons, or perhaps get a larger

training data set.

9. When you are satisfied with the network performance, click Next. Use the buttons on this screen

to save your results. You can export them into the Matlab's workspace (Save Results button) or

use the Generate M-file button to create an M-file with commands that recreate the steps that

you have just performed from the command line. Generating an M-file is a good way to learn how

to use the command-line operations of the Neural Network Toolbox software.



16/16


Kokkoras F | Paraskevopoulos K P a g e | 16

References

1. Repository Of Machine Learning Databases:

ftp://ftp.ics.uci.edu/pub/machine-learning-databases/

2. D. Harrison and D.L. Rubinfeld, "Hedonic prices and the demand for clean air", J. Environ. Eco-

nomics & Management , Vol. 5, 1978, pp.81-102.3. O. L. Mangasarian and W. H. Wolberg: "Cancer diagnosis via linear programming", SIAM News,

Volume 23, Number 5, September 1990, pp 1 & 18.

4. R.A. Fisher, "The use of multiple measurements in taxonomic problems" Annual Eugenics, 7, Part

II, pp.179-188 (1936); also in "Contributions to Mathematical Statistics" (John Wiley, NY, 1950).

5.

Matlab Help
ftp://ftp.ics.uci.edu/pub/machine-learning-databases/ftp://ftp.ics.uci.edu/pub/machine-learning-databases/ftp://ftp.ics.uci.edu/pub/machine-learning-databases/

Documents

Neural Networks Apps (v. Short, For Newbies)