A VLSI Implementable Learning Algorithm

  • View
    108

  • Download
    3

Embed Size (px)

Text of A VLSI Implementable Learning Algorithm

TABLE OF CONTENTS1. Introduction 12. Objective ............................................................................................................................. 3 1.3. Contributions of the dissertation ...................... ................................................................. S 1.4. Organization of the dissertation .......................................................................................... 6 2. Background information 2.1. On hardware implementation of neural networks ............................................................... 7 2.2. On top-down design methodology ................................................................................... 10 3. The learning algorithms 3.1. Deterministic teaming algorithm .................................................................................... 11 3.2. Original stochastic learning algorithm ............................................................................ IS 3.3 Modifications made for VLSI implementation .................................................................. 25 3.3.1. Digital Sigmoid ..................................................................................................... 27 3.3.2. Modifications of the original weight adjustment mechanism ................................. 32 3.3.3. Data representation ............................................................................................... 33 4. Top-down design methodology .................................................................................................... 37 5. Top-down design with Alopex 5.1. Choice of data format ....................................................................................................... 59 5.2. First Step: C language implementation ............................................................................. 65 5.3. Second Step: HDL functional description ......................................................................... 65 5.4. Third Step: preliminary module partitioning ..................................................................... 66 5.5. Module description 5.5.1. Weight adjustment ................................................................................................. 71 5.5.2. Output calculation units ........ : ............................................................................... 78 5.5.3. Control unit. .......................................................................................................... 83 5.5.4. Noise serial, clock generator and power-on-reset modules ..................................... 90 5.5.5. Operations and machine cycles used by the control unit and the neural array during one training iteration............................................................................... 91 5.6. Synthesis Step ................................................................................................................... 95 5.7. Placement and routing ...................................................................................................... 99 6. Conclusions and future work 6.7. Summary ........................................................................................................................ 100 6.8. Conclusions .................................................................................................................... 102 6.9. Directions for future work .............................................................................................. 106

v7. Appendix A - Software listings A.l. Perception HDL behavioral/structural descriptions & results ..................................... 107

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

A.2. Sigmoid HDL behavioral description ........................................................................ 122 A.3. Alopex C language implementation and results ......................................................... 125 A. 3. ............................................................. Alopex HDL behavioral/structural description 145 8. Appendix B B. l. .............................................................................. Design, synthesis and analysis tools 160 B. 2. Tutorials ................................................................................................................... 164 9. Bibliography ............................................................................................................................ 169 10. Author's vita... _______ .................................................................................................... ------- 172

vi

Table 5.1. Actual weight adjustments for 8 iterations ........................................................... 77

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

LIST OF TABLES

Table 5.2. Error calculations in a typical iteration ____ ............ ____ ........... _ ..................89

Table 5.3. Machine cycles for one training iteration _

_______________________________________ ............................................................91

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

viiFigure 3.1.: The Percqmon ............................................................................................... 12 Figure 3.2.: Hierarchical module partitioning ................................................................... 13 Figure 3.3.: Error/weight change probabilities ................................................................. 17 Figure 3.4.: Flowchart for Alopex C language implementation ........................................ 19 Figure 3.5.: Digital sigmoid implementation .................................................................... 25 Figure 3.6.: a) Digital sigmoid transfer function for several values of b) timing waveforms of the behavioral HDL description; c) synthesized schematic and d)chip layout .................. 30 Figure 3.7.: Integer/fractional multiplication comparison ................................................. 36 Figure4.1.: Top-down design steps ................................................................................... 40 Figure 4.2.: Sample run for results of 13 iterations of perceptron algorithm ..................... 43 Figure 4.3.: Top-level schematic ...................................................................................... 48 Figure 4.4.: Training module partitioning......................................................................... 50 Figure 4.5.: Verilog XL timing waveforms ...................................................................... 52 Figure 4.6.: Training and testing patterns ......................................................... - .............. 53 Figure 4.7.: Gate level schematic and reports generated by Synergy synthesis tools ........ 54 Figure 4.8.: Verilog XL timing waveforms of the gate level simulation ........................... 57 Figure 4.9.: Perceptron (training module) final chip layout ........................................... 58 Figure 5.1.: Verilog code for signed fractional multiplier. ................................................ 61 Figure 5.2.: Data representation for Alopex implementation ............................................. 63 Figure 5.3.: F(net) for present implementation ................................................................. 64 Figure 5.4.: HDL structural description of network architecture....................................... 67 Figure 5.5.: System block diagram ................................................................................... 70 Figure 5.6.: Timing waveforms of a weight unit updating its value .................................. 74 Figure 5.7.: Timing waveforms of output calculation units .............................................. 81 Figure 5.8.: Portion of sample run showing how neurons calculate f(net)=I Wj * Xj ....... 82 Figure 5.9.: Timing waveforms for control unit ............................................................... 86 Figure 5.10: Portion of a sample run showing actual calculated values for partial and total errors ....................'. .......................................................................................................... 89 Figure 5.11: Timing waveforms for Alopex ..................................................................... 93 Figure 5.12:Gate level schematic of output calculation unit (neuron) ............................... 96 Figure 5.13:Gate count, total area and maximum delay reports for a)weight unit, b) output calculation unit. ................................................................................................................ 97 Figure 5.14: Final layout for output calculation unit ......................................................... 99 Figure B.I.: Data flow in Cadence Design Framework II Environment .......................... 161

viii

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

Chapter 1 INTRODUCTION

1.1 MotivationDuring the last two decades, several artificial neural network architectures have been proposed to address the issue of computational intelligence (Werbos, 1974; Kohonen, 1988; Hopfield, 198S; Grossberg, 1987; Fukushima, 1984, Minsky&Papert, 1988). A fundamental characteristic of artificial neural networks is their learning capability. They implement this by performing simple calculations, such as product sums and nonlinear functions, using local operators on a high number of interconnected processing units, trying to emulate the biological neurons (Freeman, 1992). Learning is realized as various adaptation rules that define the way synaptic weights ar

Search related