Upload
electrical1012000
View
219
Download
0
Embed Size (px)
Citation preview
8/8/2019 Character Recognition Using NN
1/10
CHARACTER RECOGNITION USING ARTIFICIAL NEURAL
NETWORK
Synopsis presented by:
8/8/2019 Character Recognition Using NN
2/10
INTRODUCTION
One of the classical applications of Artificial Neural Network is Noisy Character Recognition.
Character Recognition finds its applications in a number if area, such as in banking, security
products, hospitals, evaluations of examination papers, answer sheets and even in robotics.
In this project we aim to design and implement a neural network for performing character
recognition. Because of the great flexibility in MATLABs Neural Network Toolbox, we will be
using it for the whole implementation.
Basically we are roughly dividing the whole project in three parts:
1. The first part consists of creating a simple neural network using MATLAB, thus gettingacquainted with the functions and tools that MATLAB has to offer
2. The second part will focus on character recognition, in which the difference betweentraining a network with data that is noisy or not noisy will be highlighted, (and optionallythe interpretation and response to English letters under five different noise levels).
3. The last part will deal with designing and creating a neural network in which we willfocus on how parameter changes and data manipulation can influence our design.
8/8/2019 Character Recognition Using NN
3/10
USING MATLAB FOR CHARACTER
RECOGNITION
The whole implementation can also be done even without using Neural Network Toolbox. We
have explored Image processing toolbox and found the following functions extremely helpful forour work. Following figure shows the screenshot of the MATLAB command window.
%% Manual Cropping
1. img = imread('sample.bmp');2. imshow(img)
3. imgGray = rgb2gray(img);
4. imgCrop = imcrop(imgGray);5. imshow(imgCrop)
8/8/2019 Character Recognition Using NN
4/10
Typing, help function_name on the MATLAB command window displays all the detail
regarding the function_nameFor example typing help imread displays the following:
A = imread(filename,fmt) reads a grayscale or color image from the file specified by the stringfilename, where the string fmt specifies the format of the file. If the file is not in the current
directory or in a directory in the MATLAB path, specify the full pathname of the location on
your system.
The function number two is also of great importance. IMCROP crops an image to a specified
rectangle. In the syntaxes below, IMCROP displays the input image and waits for you to specifythe crop rectangle with the mouse:
I2 = IMCROP(I)
X2 = IMCROP(X,MAP)RGB2 = IMCROP(RGB)
If you omit the input arguments, IMCROP operates on the image
in the current axes.
%% Resizing1. imgLGE = imresize(imgCrop, 5, 'bicubic');
2. imshow(imgLGE)
%% Rotation
1. imgRTE = imrotate(imgLGE, 35);
2. imshow(imgRTE)
%% Binary Image
1. imgBW = im2bw(imgLGE, 0.90455);2. imshow(imgBW)
IM2BW Convert image to binary image by thresholding.
IM2BW produces binary images from indexed, intensity, or RGB images. To do this, it
converts the input image to grayscale format (if it is not already an intensity image), and then
converts this grayscale image to binary by thresholding. The output binary image BW has valuesof 1 (white) for all pixels in the input image with luminance greater than LEVEL and 0 (black)
for all other pixels. (Note that you specify LEVEL in the range [0,1], regardless of the class ofthe input image.)
1. BW = IM2BW(I,LEVEL) converts the intensity image I to black and white.
2. BW = IM2BW(X,MAP,LEVEL) converts the indexed image X with colormap MAP
to black and white.
8/8/2019 Character Recognition Using NN
5/10
3. BW = IM2BW(RGB,LEVEL) converts the RGB image RGB to black and white.
AN INTRODUCTION TO NEURAL NETWORK
What is an Artificial Neural Network?
An Artificial Neural Network (ANN) is an information-processing paradigm that is inspired by
the way biological nervous systems, such as the brain, process information. The key element of
this paradigm is the novel structure of the information processing system. It is composed of alarge number of highly interconnected processing elements (neurones) working in unison to
solve specific problems. ANNs, like people, learn by example. An ANN is configured for a
specific application, such as pattern recognition or data classification, through a learning process.Learning in biological systems involves adjustments to the synaptic connections that existbetween the neurones. This is true of ANNs as well.
Why use neural networks?
Neural networks, with their remarkable ability to derive meaning from complicated or imprecise
data, can be used to extract patterns and detect trends that are too complex to be noticed by either
humans or other computer techniques. A trained neural network can be thought of as an "expert"
in the category of information it has been given to analyse. This expert can then be used toprovide projections given new situations of interest and answer "what if" questions.
Other advantages include:
1. Adaptive learning: An ability to learn how to do tasks based on the data given for training
or initial experience.2. Self-Organisation: An ANN can create its own organisation or representation of the
information it receives during learning time.
3. Real Time Operation: ANN computations may be carried out in parallel, and specialhardware devices are being designed and manufactured which take advantage of this
capability.
4. Fault Tolerance via Redundant Information Coding: Partial destruction of a networkleads to the corresponding degradation of performance. However, some network
capabilities may be retained even with major network damage.
Neural networks versus conventional computers
Neural networks take a different approach to problem solving than that of conventional
computers. Conventional computers use an algorithmic approach i.e. the computer follows a setof instructions in order to solve a problem. Unless the specific steps that the computer needs to
follow are known the computer cannot solve the problem. That restricts the problem solving
capability of conventional computers to problems that we already understand and know how to
8/8/2019 Character Recognition Using NN
6/10
solve. But computers would be so much more useful if they could do things that we don't exactly
know how to do.
Neural networks process information in a similar way the human brain does. The network iscomposed of a large number of highly interconnected processing elements(neurones) working in
parallel to solve a specific problem. Neural networks learn by example. They cannot beprogrammed to perform a specific task. The examples must be selected carefully otherwise
useful time is wasted or even worse the network might be functioning incorrectly. Thedisadvantage is that because the network finds out how to solve the problem by itself, its
operation can be unpredictable.
On the other hand, conventional computers use a cognitive approach to problem solving; the way
the problem is to solved must be known and stated in small unambiguous instructions. Theseinstructions are then converted to a high level language program and then into machine code that
the computer can understand. These machines are totally predictable; if anything goes wrong is
due to a software or hardware fault.
Neural networks and conventional algorithmic computers are not in competition but complement
each other. There are tasks are more suited to an algorithmic approach like arithmetic operations
and tasks that are more suited to neural networks. Even more, a large number of tasks, require
systems that use a combination of the two approaches (normally a conventional computer is usedto supervise the neural network) in order to perform at maximum efficiency.
These features of Neural Network motivates us for carrying out character recognition using
Neural network architectures.
Training Algorithms for Neural Network
Once a network has been structured for a particular application, that network is ready to betrained. To start this process the initial weights are chosen randomly. Then, the training, or
learning, begins.
There are two approaches to training - supervised and unsupervised. Supervised training involves
a mechanism of providing the network with the desired output either by manually "grading" thenetwork's performance or by providing the desired outputs with the inputs. Unsupervised training
is where the network has to make sense of the inputs without outside help.
The vast bulk of networks utilize supervised training. Unsupervised training is used to perform
some initial characterization on inputs. However, in the full blown sense of being truly selflearning, it is still just a shining promise that is not fully understood, does not completely work,
and thus is relegated to the lab.
Supervised Training.
In supervised training, both the inputs and the outputs are provided. The network then processes
the inputs and compares its resulting outputs against the desired outputs. Errors are then
propagated back through the system, causing the system to adjust the weights which control the
8/8/2019 Character Recognition Using NN
7/10
network. This process occurs over and over as the weights are continually tweaked. The set of
data which enables the training is called the "training set." During the training of a network the
same set of data is processed many times as the connection weights are ever refined.
The current commercial network development packages provide tools to monitor how well an
artificial neural network is converging on the ability to predict the right answer. These toolsallow the training process to go on for days, stopping only when the system reaches some
statistically desired point, or accuracy. However, some networks never learn. This could bebecause the input data does not contain the specific information from which the desired output is
derived. Networks also don't converge if there is not enough data to enable complete learning.
Ideally, there should be enough data so that part of the data can be held back as a test. Manylayered networks with multiple nodes are capable of memorizing data. To monitor the network to
determine if the system is simply memorizing its data in some nonsignificant way, supervised
training needs to hold back a set of data to be used to test the system after it has undergone itstraining. (Note: memorization is avoided by not having too many processing elements.)
If a network simply can't solve the problem, the designer then has to review the input andoutputs, the number of layers, the number of elements per layer, the connections between the
layers, the summation, transfer, and training functions, and even the initial weights themselves.Those changes required to create a successful network constitute a process wherein the "art" of
neural networking occurs.
Another part of the designer's creativity governs the rules of training. There are many laws
(algorithms) used to implement the adaptive feedback required to adjust the weights duringtraining. The most common technique is backward-error propagation, more commonly known as
back-propagation. These various learning techniques are explored in greater depth later in this
report.
Yet, training is not just a technique. It involves a "feel," and conscious analysis, to insure that thenetwork is not overtrained. Initially, an artificial neural network configures itself with the general
statistical trends of the data. Later, it continues to "learn" about other aspects of the data which
may be spurious from a general viewpoint.
When finally the system has been correctly trained, and no further learning is needed, theweights can, if desired, be "frozen." In some systems this finalized network is then turned into
hardware so that it can be fast. Other systems don't lock themselves in but continue to learn
while in production use.
Unsupervised, or Adaptive Training.The other type of training is called unsupervised training. In unsupervised training, the network
is provided with inputs but not with desired outputs. The system itself must then decide whatfeatures it will use to group the input data. This is often referred to as self-organization or
adaption.
At the present time, unsupervised learning is not well understood. This adaption to the
environment is the promise, which would enable science fiction types of robots to continually
8/8/2019 Character Recognition Using NN
8/10
learn on their own as they encounter new situations and new environments. Life is filled with
situations where exact training sets do not exist. Some of these situations involve military action
where new combat techniques and new weapons might be encountered. Because of thisunexpected aspect to life and the human desire to be prepared, there continues to be research
into, and hope for, this field. Yet, at the present time, the vast bulk of neural network work is in
systems with supervised learning. Supervised learning is achieving results.
One of the leading researchers into unsupervised learning is Tuevo Kohonen, an electricalengineer at the Helsinki University of Technology. He has developed a self-organizing network,
sometimes called an auto-associator, that learns without the benefit of knowing the right answer.
It is an unusual looking network in that it contains one single layer with many connections. Theweights for those connections have to be initialized and the inputs have to be normalized. The
neurons are set up to compete in a winner-take-all fashion.
Kohonen continues his research into networks that are structured differently than standard,
feedforward, back-propagation approaches. Kohonen's work deals with the grouping of neurons
into fields. Neurons within a field are "topologically ordered." Topology is a branch ofmathematics that studies how to map from one space to another without changing the geometric
configuration. The three-dimensional groupings often found in mammalian brains are anexample of topological ordering.
Kohonen has pointed out that the lack of topology in neural network models make today's neural
networks just simple abstractions of the real neural networks within the brain. As this research
continues, more powerful self-learning networks may become possible.
Backpropagation Learning Algorithm
The backpropagation algorithm trains a given feed-forward multilayer neural network for a givenset of input patterns with known classifications. When each entry of the sample set is presented
to the network, the network examines its output response to the sample input pattern. The output
response is then compared to the known and desired output and the error value is calculated.Based on the error, the connection weights are adjusted. The backpropagation algorithm is based
on Widrow-Hoff delta learning rule in which the weight adjustment is done through meansquare errorof the output response to the sample input. The set of these sample patterns arerepeatedly presented to the network until the error value is minimized.
Refer to the figure 2 below that illustrates the backpropagation multilayer network with
layers. represents the number of neurons in th layer. Here, the network is presented the th
pattern of training sample set with -dimensional input and -
dimensional known output response . The actual response to the input pattern
by the network is represented as . Let be the output from the th neuron
8/8/2019 Character Recognition Using NN
9/10
in layer for th pattern; be the connection weight from th neuron in layer to
th neuron in layer ; and be the error value associated with the th neuron in layer .
Figure : Backpropagation Neural Network
Steps to follow until error is suitably small
Step 1: Input training vector.
Step 2: Hidden nodes calculate their outputs.Step 3: Output nodes calculate their outputs on the basis of Step 2.
Step 4: Calculate the differences between the results of Step 3 and targets.
Step 5: Apply the first part of the training rule using the results of Step 4.
Step 6: For each hidden node, n, calculate d(n).Step 7: Apply the second part of the training rule using the results of Step 6.
8/8/2019 Character Recognition Using NN
10/10
Steps 1 through 3 are often called theforward pass, and steps 4 through 7 are often called thebackward pass. Hence, the name: back-propagation.
Conclusion:
Our preliminary research in noisy character recognition clearly indicates that Neural Network
can be a most important tool for the implementation of the Character Recognition System.
Further the latest CAD tools such as MATLAB, can provide us with greater flexibility andenhance the efficiency and productivity of the developers by reducing the complexity of coding.