A Practical Approach to Introducing Electrical Undergraduates to Artificial Neural Networks

8/3/2019 A Practical Approach to Introducing Electrical Undergraduates to Artificial Neural Networks

1/9

A practical approach to introducing electricalundergraduates to artificial neural networksInternational Journal of Electrical Engineering Education,Apr 2009byHalkiadis, I S

Abstract

Artificial intelligence techniques are increasingly being applied to a variety of problems inthe field of electrical engineering, and consequently universities worldwide are paying moreattention to training electrical engineers in applying neural networks (ANNs) in practice. Aspart of the above training, an introductory practical project on ANNs has been offered as anelective to the undergraduates in the author's department. The project introduces a singlelayer ANN model consisting of two neurons. The algorithm, the hardware implementationand the test procedure concerning one of the neurons, are described. Then the studentsapply the same techniques for the other neuron and combine the two circuits to form thefinal ANN hardware. The agreement of the results conducted by the theoretical models withthose obtained by the constructed circuits, allow the students to become familiar with ANNhardware techniques and realise that they present a practical and efficient solution.

Keywords ANN; electrical engineering education; neural network hardware

(ProQuest: ... denotes formulae omitted.)

Artificial neural networks (ANNs), also called simply neural networks, are informationprocessing systems with their design inspired by studies of the ability of the human brain tolearn from observations and to generalise by abstraction.1 The fact that ANNs can betrained to learn any arbitrary nonlinear input-output relationships from corresponding datahas resulted, especially in recent years, in their extensive and successful use in a broad

spectrum of electrical engineering applications, such as: power systems,2-5 high voltagetechnology,6-8 communications,9-11 automatic control and robotics,12-14 signalprocessing,15-17 circuits and devices,18-20 and electrical machines.21-23

In view of the progress of development of artificial neural networks for electrical engineeringapplications, the practical use of neural networks has been gaining more attention thanbefore in universities24-28 and in industry.29-30 In practice, there are two basic ways ofimplementing neural networks: a) in software simulations based on the theory of ANNs andrun in conventional computers, and b) in hardware where signals are processed by circuits.In engineering courses students learn best when they are given the opportunity to applytheoretical concepts to real-world systems and ANN hardware implementation presentsattractive possibilities for giving them such an opportunity. In the author's department,undergraduates are overloaded with many subjects and so conducting many ANN design

experiments is not possible. In view of such constraints, a practical short ANN project hasbeen designed which is offered as an elective to undergraduates who have been taught thebasics of analogue electronics but are without any previous knowledge of ANNs. Althoughprogramming is certainly basic in engineering courses, most of the students who are takingthis introductory neural network project are supplied with a variable software background.Therefore instead of introducing the students to any particular executable language, thefocus of this project is on the basic algorithmic concepts and constructs in such a way thatthe developed ANN is not required to be programmable on line.
http://findarticles.com/p/articles/mi_qa3792/http://findarticles.com/p/articles/mi_qa3792/http://findarticles.com/p/articles/mi_qa3792/is_200904/http://findarticles.com/p/articles/mi_qa3792/is_200904/http://findarticles.com/p/articles/mi_qa3792/is_200904/http://findarticles.com/p/search/?qa=Halkiadis,%20I%20Shttp://findarticles.com/p/search/?qa=Halkiadis,%20I%20Shttp://findarticles.com/p/search/?qa=Halkiadis,%20I%20Shttp://findarticles.com/p/search/?qa=Halkiadis,%20I%20Shttp://findarticles.com/p/articles/mi_qa3792/is_200904/http://findarticles.com/p/articles/mi_qa3792/


2/9

The requirements for such a project can be summarised in the following objectives:

1 It should be easy to construct on a simple breadboard;

2 All the electronic components used must be commercially available;

3 Suggested procedure should be handy and applicable. Difficult tasks should be avoided;

4 The input and output signals can be currents or voltages; and

5 All technical information should be related to students' prior knowledge.

The first three objectives will be able to smoothen the diverse background knowledge,address the diversity of prior hardware experience and will enable experimental work withinpre-existing facilities. The fourth reflects the need to use the proposed model on realprojects. Finally, the fifth goal focuses on immediate feedback, speeding up theexperimental work. To master the technique, the students start by learning about neurons

and algorithms but soon find themselves testing an ANN circuit that they themselves havebuilt.

The proposed approach that is presented in this paper involves:

* presentation of the primary reading material for the course, since it is referred to studentswho have no previous knowledge of the subject;

* planning of the algorithm which describes the proposed ANN;

* learning process of the above algorithm;

* development of the proposed ANN in hardware;

* comparison of the results taken by the theoretical approach with those obtainedpractically.

The biological neuron

The fundamental processing element of a biological system is the neuron like the oneillustrated in Fig. 1. The dendrites receive signals from the other neurons, the cell bodyprovides the support function for proccessing them and the axon is the output mechanismthat conducts signals away from the cell to the other neurons via the interconnection points,the synapses.

Artificial Neural Networks (ANNs)

An ANN is architecturally similar to a biological system in that the ANN also uses a numberof simple, interconnected artificial neurons. In a restricted sense artificial neurons aresimple emulations of biological neurons. Figure 2 shows the mathematical modelimplemented by an artificial neuron k with n inputs, denoted as X1, X2, . . . , Xn. Theseinputs can be binary, bipolar or continuous. Each line connecting these inputs to the neuron


3/9

is assigned a weight, denoted as W1, W2, . . . , Wn, respectively. They can be assigned anyvalue. Weights represent the strength of connection between two processing elements. Theaction, which determines whether the neuron is to be active or not, is given by theformula:31

... (1)

The output of the neuron is a function of its action:

k=(uk) (2)

The above equations indicate that the neuron will collect data from other neurons throughweighted inputs, calculate the total input, and then would decide to turn on or not, based onsome function indicated here by .(uk). A simple example of such a function is the thresholdfunction described in Fig. 3 that passes information only when the sum of the weightedinputs exceeds the threshold value T.32

Neural networks must 'learn' how to process input before they can be utilised in anapplication. The process of training a neural network involves adjusting the input weights oneach neuron such that the output of the network is consistent with the desired output. Theadjustment of the weights will be done through a learning algorithm. The simplest and oneof the most studied is the Hebbian learning algorithm which is described Ref. 33 as havingsix steps shown in Table 1.

The ANN op amp hardware model

Equations (1) and (2) show that when the sum of the inputs of a neuron reaches a certainvalue than it will react.

Conventional op amp circuits can implement these processing functions as shown in Fig. 4where the electronic model of the neuron is depicted. It can be observed that a neuronconsists of weight amplifiers with fixed gains. Inputs are voltage signals and each gainpresents an input weight and its value is set according to the values calculated by theHebbian algorithm. A summing amplifier follows and then a comparator where the output isset at one of two levels, depending on whether the total input uk is greater or less thansome threshold value Vthr. An electronic ANN consists of a number of these neuronsconnected together.

Overview of the project

The proposed ANN

After students have been familiar with the basic theory, they will be involved in thehardware implementation of a single layered31 ANN consisted by two neurons 'A' and 'B',with two binary inputs V1 and V2 and two outputs out1 and out2. The architecture of thatANN is illustrated in Fig. 5. The students will be asked to adjust the weights and thresholdsof the two neurons so that out1 corresponding to 'neuron A' will be 1 when both inputs are1, zero otherwise and out2 corresponding to 'neuron B' will be zero if both inputs are zeros,1 otherwise. The response of the two outputs is shown in Table 2.


4/9

Proposed method

For a successful approach to design the final ANN hardware, the following method isrecommended:

a) Plan the Hebbian algorithm for 'neuron A' in order to adjust the weights;

b) Specify the appropriate threshold function for 'neuron A';

c) Test 'neuron A' using the above results;

d) Implement 'neuron A' in hardware;

e) Test the implemented hardware;

f) Follow all above steps for 'neuron B';

g) Combine the developed circuits of both neurons to form the final ANN hardware.

To prevent students from making mistakes and to save supervisors' time correcting them,steps a-g of the above proposed method can be described analytically for 'neuron A'. Thenthe students will be asked to apply the same steps for 'neuron B'. Finally they should attachboth neurons together according to Fig. 5, to form the final circuit and make sure that theyare getting the appropriate results by testing it.

Planning of the algorithm for 'neuron A'

Following the Hebbian rule steps, described above, the algorithm is formed as shown below:

- inputs: j = 2,

- initialising weights to zero: W1 = 0, W2 = 0,

- set activation values for the first set of inputs (shown in Table 2): V1 = 0, V2 = 0,

- set activation for desired output corresponding to first set: t = 0,

- modify all connections Wi according to Wj = Vjt : W1 = 0.0 = 0, W2 = 0.0 = 0,

- adjust the weights as Wj (new) = Wj (old) Wj : W1 = 0 0 = 0, W2 = 0 0 = 0,

Similarly the above steps are repeated for the three other input sets of Table 2. The weightsare updated when all patterns of the input pairs have been presented. The details are givenin Table 3.

As can be seen, the final values for weights W1 and W2 have ended up, through thelearning process, to W1 = 1 and W2 = 1. Adjusted weights end up with the same valuesdespite the rotation of input pairs i.e. (1, 1)-(1,0)-(0,1)-(0,0) or other.


5/9

Specifying the appropriate threshold function for 'neuron A'

Since the outputs of this neuron take binary values, the threshold function described in Fig.3 is chosen. The value of T can be defined from the threshold parameter uk T, whichcombined with (1) as formed for this neuron, gives T V1W1 V2W2. Using the highestvalues for V1, V2 we have T 1 E1 1 E1 = 2. We choose T = 1.5 and the threshold function

for 'neuron A' can finally be formed as:

...

Testing of 'neuron A'

The network can be tested by using (1), the estimated values W1 = 1, W2 = 1 and thethreshold function shown above. The results are illustrated in Table 4. As can be seen allproduced results are correct i.e. are in agreement with the desired output t values shown inTable 3.

Development of the hardware model for 'neuron A'

Based on the model depicted in Fig. 4 and applying all circuit restrictions mentioned in theintroduction, the schematic circuit of 'neuron A' is given in Fig. 6, where the inputs, theweights, the summing junction and the threshold function sections can easily bedistinguished. As can be seen the circuit consists only of resistors and standard unbalanced741 op-amps and can be easily constructed in a breadboard.

Testing 'neuron A' circuit

To test the circuit, a 4-channel Gould 4068 digital oscilloscope to record the inputoutputsignals and two Instec GFG 8250A function generators for the input signals, were used. All

those can be seen in Fig. 9 where a photo of the experimental set-up for the final circuit isshown. Figure 7 shows V1 and V2 input pulses and the corresponding out1 of theimplemented hardware 'neuron A' model. As can easily be seen the output of the circuit isformed with a complete agreement with the output of the corresponding theoretical modeli.e. is 1 when both inputs are 1, zero otherwise.

Development of hardware for 'neuron B' and final model

By applying the same methods the students are now asked to implement and test thehardware model of 'neuron B'. By doing so they will soon find out that they develop thesame hardware as that of 'neuron A', but with different weight values of W1 = 2 and W2 =2. Then, following the layout of Fig. 5, they have to interconnect both hardware models toimplement the final requested hardware neural model, the circuit of which should be similarto that shown in Fig. 8. Finally the students are asked to test the functionality of theimplemented analogue circuit. Figure 9 illustrates the experimental set up for testing thefinal circuit from where the construction of the circuit on the breadboard can be seen. Figure10 depicts all input-output signals as plotted by the oscilloscope plotter. As can easily beconcluded after observing the plotted signals, the response of both outputs is in completeagreement with those requested by the project and shown in Table 2.

Conclusion


6/9

A practical introductory ANN project has been designed for the purpose of ANN education atthe undergraduate electrical engineering level. The project focuses on the basic algorithmicconcepts of the designed ANNs and their hardware implementation. In this project thestudents had the opportunity to design a) theoretical models and b) models implemented inanalogue hardware, of two ANNs, and to justify that in all cases theoretical and hardwaremodels produce the same results. By combining the two hardwared ANNs students learned

the way to connect different ANNs in order to achieve a more complicated task. Thepractical approach adopted in this project, by bringing the theoretical models of ANNs intothe practical applications, has achieved its objective of making ANNs more familiar andattractive to electrical engineering undergraduates.

References

1 Q.-J. Zhang, K. C. Gupta and V. K. Devabhaktuni, 'Artificial neural networks for RF andmicrowave design - from theory to practice', IEEE Trans. Microwave Theory and Techniques,51(4/2) (2003), 1339-1350.

2 A. M. Stankovic, A. T. Saric and M. Milosevic, 'Identification of nonparametric dynamic

power system equivalents with artificial neural networks', IEEE Trans. Power Systems, 18(4)(2003), 1478-1486.

3 A. Keyhani et al., 'Composite neural network load models for power system stabilityanalysis', in Proc. PSCE 2004, IEEE Power System Conference & Exploration, October 10-13,2004, New York (IEEE, New York, 1994), Poster B.62.

4 P. Salmern and J. R. Vzquez, 'Practical design of a three-phase active power-lineconditioner controlled by artificial neural networks', IEEE Trans. Power Delivery,20(2)(2005), 1037-1044.

5 G. J. Tsekouras, N. D. Hatziargyriou and E. N. Dialynas, 'An optimised adaptive neural

network for annual midterm energy forecasting', IEEE Trans. Power Systems, 21(1) (2006),385-391.

6 A. L. O. Fernandez and N. K. I. Ghonaim, A novel approach using a FIRANN for faultdetection and direction estimation for high-voltage transmission lines, IEEE Trans. PowerDelivery, 17(4) (2002), 894-900.

7 T. Bouthiba, 'Fault location in EHV transmission lines using artificial neural networks', Int.J. Appl. Comput. Sci., 14(1) (2004), 69-78.

8 S. Happe H.-G. Kranz and W. Krause, 'Advanced suppression of stochastic pulse shapedpartial discharge disturbances', Trans. Dielectrics and Electrical Insulation, 12(2) (2005),

265-275.

9 Kim Namjin, N. Kehtarnavaz, M. B. Yeary and S. Thornton, DSP-based hierarchical neuralnetwork modulation signal classification, IEEE Trans. Neural Networks, 14(5) (2003), 1065-1071.

10 J. Choi, A. C. De C. Lima and S. Haykin, 'Filter-trained recurrent neural equalizers fortime-varying channels' IEEE Trans. Communications, 53(3) (2005), 472-480.


7/9

11 A. Patnaik, D. Anagnostou and C. G. Christodoulou, 'Neural networks in antennaengineering - beyond black-box modeling', in Proc. IEEE/ACES International Conference onWireless Communications and Applied Computational Electromagnetics, Special Session:Communication Antenna Analysis and Design, Honolulu, Hawaii, April 3-7, 2005, pp. 598-601.

12 G. van Schoor, J. D. van Wyk and I. S. Shaw, 'Optimal control of a hybrid powercompensator using an artificial neural network controller', Trans. Industry Applications,38(2) (2002), 467- 475.

13 B. Widrow and M. Malini Lamego, 'Neurointerfaces', IEEE Trans. Control SystemsTechnology, 10(2) (2002), 221-228.

14 S. Dhillon and K. W. Horch, 'Direct neural sensory feedback and control of a prostheticarm', Trans. Neural Systems and Rehabilitation Engineering, 13(4) (2005), 468- 472.

15 D. King, C. Flanagan, W. B. Lyons and E. Lewis, 'An optical fiber water sensor utilizingsignal processing techniques and artificial neural network pattern recognition', in Proc. 2002

IEEE Intl Conf. on Sensors, 12-14 June 2002, Florida, USA, Vol. 2 (IEEE, Piscataway, NJ,2002), pp. 1374-1378.

16 F. Cau, A. Fanni, A. Montisci, P. Testoni and M. Usai, 'Artificial neural networks fornondestructive evaluation with ultrasonic waves in not accessible pipes', Conference Recordof the 2005 Industry Applications Conference, 40th IAS Annual Meeting, 2-6 Oct. 2005, Vol.1 (IAS, 2005), pp. 685-692.

17 S. Mare, 'Detection of nonlinear distortion in audio signals', IEEE Trans. Broadcasting,48(2) (2002), 76-80.

18 R. Harrison, 'A low-power low-noise CMOS amplifier for neural recording applications', J.Solid- State Circuits, 38(6) (2003), 958-965.

19 C. T. Yen, Wan-de Weng and Y. T. Lin, 'FPGA realization of a neural-network-basednonlinear channel equalizer', IEEE Trans. Industrial Electronics, 51(2) (2004), 472-479.

20 S.-J. Hsieh, 'Artificial neural networks and statistical modelling for electronic stressprediction using thermal profiling', IEEE Trans. Electronics Packaging and Manufacturing,27(1) (2004), 49-58.

21 B. Li, Mo-Yuen Chow, Y. Tipsuwan and J. C. Hung, 'Neural-network-based motor rollingbearing fault diagnosis', Trans. Industrial Electronics, 47(5) (2000), 1060-1069.

22 J. Lehtonen and H. N. Koivo, 'Fault diagnosis of induction motors with dynamical neuralnetworks', Proc. 2005 IEEE Intl Conf. on Systems, Man & Cybernetics, Hawaii, USA, October10-12, 2005 (IEEE., Piscataway, NJ, 2005), pp. 2979-2984.

23 B. Karanayil, M. F. Rahman and C. Grantham, 'Stator and rotor resistance observers forinduction motor drive using fuzzy logic and artificial neural networks', IEEE Trans. EnergyConversion, 20(4) (2005), 771-780.


8/9

24 Curriculum 2001 Joint Task Force, Computing Curricula 2001 Computer Science: FinalReport, December 15, 2001, Los Alamitos, CA, availableathttp://www.computer.org/portal/cms_docs_ieeecs/ieeecs/education/cc2001/cc2001.pdf.

25 F. Jurado, M. Castro and J. Carpio, 'Experiences with fuzzy logic and neural networks in acontrol course', IEEE Trans. Education, 45(2) (2002), 161-167.

26 S. P. Imberman, 'An intelligent agent approach for teaching neural networks usingLEGO handy board robots', J. Educ. Resources in Computing (JERIC), 4(3) (2004), 1-12.

27 Russell et al., 'Unifying an Introduction to Artificial Intelligence Course through MachineLearning Laboratory 4. Experiences', in Proc. 2005 ASEE Annual Conference & Exposition,June 20-23, 2004, Salt Lake City, USA, availableathttp://soa.asee.org/paper/conference/paper_view. cfm?id=21336.28 T. L. Hemminger, 'Understanding Transmission Line Impedance Matching Using NeuralNetworks and Power Point', in Proc. 35th ASEE/IEEE Frontiers in Education Conference,section T4E-18, Oct. 19-22, 2005, Indianapolis, USA, availableathttp://ieeexplore.ieee.org/stamp/stamp.jsp? arnumber=1611978&isnumber=33854.

29 'NEuroNet Roadmap 2001, 3. Future Prospects for Neural Networks, 3.3.1. Tools - NeuralNetworks Hardware', available at:http://www.kcl.ac.uk/neuronet/about/roadmap/-hardware.html.

30 M. R. G. Meireles, P. E. M. Almeida and M. G. Simes, 'A Comprehensive Review forIndustrial Applicability of Artificial Neural Networks', IEEE Trans. Industrial Electronics,50(3) (2003), 585-601.

31 S. Haykin, Neural Networks (Prentice-Hall, Upper Saddle River, NJ, 1999), p. 11.

32 L. H. Csoukalas and R. E. Uhric, Fuzzy and Neural Approaches in Engineering (John Wiley,New York, 1977), p. 195.

33 A. Upegui, C. A. Pea-Reyes and E. Snchez, A Hardware Implementation of a Networkof Functional Spiking Neurons with Hebbian Learning, presented at BIOADIT (BiologicalInspired Approaches to Advanced Information Technology), 1st International Workshop,January 29-30, 2004, Lausanne, Switzerland, available athttp://reds.heig-vd.ch/contacts/andres_upegui/papers/UpeguiHebbian04.pdf.

I. S. Halkiadis

Department of Electrical Engineering, Technological Institution of Chalkida, Psahna, Greece

E-mail:[email protected]

Copyright Manchester University Press Apr 2009Provided by ProQuest Information and Learning Company. All rights Reserved

Bibliography for: "A practical approach to introducingelectrical undergraduates to artificial neural networks"
http://www.computer.org/portal/cms_docs_http://www.computer.org/portal/cms_docs_http://www.computer.org/portal/cms_docs_http://soa.asee.org/paper/conference/paper_viewhttp://soa.asee.org/paper/conference/paper_viewhttp://soa.asee.org/paper/conference/paper_viewhttp://ieeexplore.ieee.org/stamp/stamp.jsphttp://ieeexplore.ieee.org/stamp/stamp.jsphttp://ieeexplore.ieee.org/stamp/stamp.jsphttp://www.kcl.ac.uk/neuronet/about/roadmap/-hardware.htmlhttp://www.kcl.ac.uk/neuronet/about/roadmap/-hardware.htmlhttp://www.kcl.ac.uk/neuronet/about/roadmap/-hardware.htmlhttp://www.kcl.ac.uk/neuronet/about/roadmap/-hardware.htmlhttp://reds.heig-vd.ch/contacts/andres_upegui/papers/http://reds.heig-vd.ch/contacts/andres_upegui/papers/http://reds.heig-vd.ch/contacts/andres_upegui/papers/http://reds.heig-vd.ch/contacts/andres_upegui/papers/mailto:[email protected]:[email protected]:[email protected]:[email protected]://reds.heig-vd.ch/contacts/andres_upegui/papers/http://reds.heig-vd.ch/contacts/andres_upegui/papers/http://www.kcl.ac.uk/neuronet/about/roadmap/-hardware.htmlhttp://www.kcl.ac.uk/neuronet/about/roadmap/-hardware.htmlhttp://ieeexplore.ieee.org/stamp/stamp.jsphttp://soa.asee.org/paper/conference/paper_viewhttp://www.computer.org/portal/cms_docs_


9/9

Halkiadis, I S "A practical approach to introducing electrical undergraduates to artificialneural networks". International Journal of Electrical Engineering Education. FindArticles.com.05 Sep, 2009. http://findarticles.com/p/articles/mi_qa3792/is_200904/ai_n32422883/
http://findarticles.com/p/articles/mi_qa3792/is_200904/ai_n32422883/http://findarticles.com/p/articles/mi_qa3792/is_200904/ai_n32422883/http://findarticles.com/p/articles/mi_qa3792/is_200904/ai_n32422883/http://findarticles.com/p/articles/mi_qa3792/is_200904/ai_n32422883/http://findarticles.com/p/articles/mi_qa3792/is_200904/ai_n32422883/http://findarticles.com/p/articles/mi_qa3792/is_200904/ai_n32422883/

Documents

A Practical Approach to Introducing Electrical Undergraduates to Artificial Neural Networks