Desire Gettingstarted

8/3/2019 Desire Gettingstarted

1/30

David Scuse September, 20111

DESIRE NEURAL NETWORK SYSTEM

Introduction:

The Desire system is a neural network modelling tool that allows the user to build models ofneural networks. Using the system, almost any neural network model can be developed,including those that require differential equations. This is in contrast to conventional neuralnetwork tools that contain a variety of pre-defined models; the user can modify a modelsparameters but can not define new models (i.e. the system is a black box). With the Desiresystem, the neural network is defined using statements in the Desire programming language sothe user can see how each portion of the neural network performs its processing and, if necessary,can modify the processing. The Desire language is a high-level language so the user does not getlost in the details (as happens with neural network toolkits written in C/C++).

Running the DesireW (Windows) System

The Desire system (currently version 15.0) runs under Windows and Linux. (The followinginstructions refer to the Windows version.) To begin, double-click the command fileDesireW.bat to launch the Desire system. This batch file opens a Desire editor window and theDesire command window.

Figure 1: The Desire Editor Window and the Desire Command Window


2/30


Load a file into the Editor window either by dragging and dropping it onto the editor windowor by using the editors Open command). You may store your files in the same folder as theDesire system or in any other convenient folder. Do not use Windows to associate Desire sourcefiles (.src and .lst) with the Desire Editor doing so causes the OK button in the Editorwindow to stop functioning correctly.

Figure 2: Loading A Desire Source File


3/30


Transfer the file to Desire by clicking on the OK button in the editor window (or using the

shortcut alt-T O). Then type the command erun (or zz) into the Desire command window.This command causes the program to be executed.

Figure 3: Transferring A Desire Source File to the Desire Command Window


4/30


5/30


Figure 5: Graph Window Menu Items

Similarly, the standard Windows facilities for copying and pasting the contents of the Desire

command window are available by clicking the top left icon of the window.

Figure 6: Command Window Menu Items

By modifying the Properties of the command window, you can switch from the default white texton a black background to black text on a white background.

Source Files

Normally, Desire source files have the extension .src. Files with the .src extension do not

contain line numbers while numbered source files have the extension .lst. If a source filealready contains line numbers, the line numbers can be removed by using the Desire commandwindow command keep 'xxx'to create an unnumbered version of the file on disk (the file willbe named xxx.src). Internally, Desire still keeps a file numbered so that it can generate errormessages and so that the programmer can refer to individual lines.

In general, unnumbered source files are preferable to numbered source files in Desire.


6/30


To view the line numbers used in the Desire command window for the current source file, typethe Desire command list+. This causes the file to be listed with line numbers in the commandwindow. To list a specific line, type list+ 2200 (or, for a range of lines, type list+ 2200-2300) in the Desire command window. To save a copy of the current file with line numbers, typelist+ 'filename.lst' This causes a copy of the file to be saved in the Desire directory (which

is not necessarily the same directory as the original source file). If you want to save the .lstfilein a different directory, add the complete path to the file in the command; for example, list+'D:\MyFolder\filename.lst'

A Quick Tour: Learning the OR Patterns

The following sections briefly describe a simple single-layer, feedforward Desire neural network.The SLFF1 program in Figure 7 defines a neural network that attempts to learn the logical OR

patterns ({0,0} {0}, {1,1} {1}, etc.). The SLFF1 program defines a 2-1 network ofthreshold units and uses the delta training rule to learn the patterns. The output of the SLFF1program is shown in the Graph Window in Figure 8. Each of the generated symbols ( )

represents the value of the tss (the sum of squares of the error) at a particular point in time. Thescreen is organized in a graph format: the vertical axis represents the tss and ranges from -scale (represented by -) at the bottom to +scale (represented by +) at the top. The value of scaleis defined in the program and is displayed at the bottom of the screen. In this example, thevertical axis ranges from -4 at the bottom to +4 at the top. The horizontal axis represents thetime, t, and begins at 1 on the left and ranges to the total number of training iterations on theright. The number of iterations of the program is the number of training epochs times thenumber of patterns. A training epoch is the time period during which each pattern is presentedonce to the network. The number of training epochs is defined in the variable Nepochs. In theprogram below, the number of program iterations is 100 (25 * 4). The horizontal axis on thegraph displays the values of time, t, beginning at 1 and continuing for 100 iterations.

------------------------------------------------------ Single-Layer Feedforward Network (SLFF1)-- activation function: threshold-- learning rule: delta learning--------------------------------------------------Npat=4 | Ninp=2 | Nout=1ARRAY Layer1[Ninp],Layer2[Nout],Weights[Nout,Ninp]ARRAY Target[Nout],Error[Nout]ARRAY INPUT[Npat,Ninp],TARGET[Npat,Nout]---------------------------------------------------- Define the OR training patterns--------------------------------------------------data 0,0;1,0;0,1;1,1 | read INPUTdata 0;1;1;1 | read TARGET---------------------------------------------------- Initialize Run-Time Variables

--------------------------------------------------scale=4 | display R | display N-16 | display W 362,0Lrate=0.2 | Nepochs=25---------------------------------------------------- Learn the weights--------------------------------------------------tss=0t=1 | NN=Nepochs*Npat | TMAX=NN-1drunSTOP--------------------------------------------------DYNAMIC


7/30


---------------------------------------------------- The following statements learn the weights-- The statements are executed once for each training pattern-- for a total of Nepochs * Npat learning repetitions--------------------------------------------------iRow=tVECTOR Layer1=INPUT#VECTOR Target=TARGET#VECTOR Layer2=swtch(Weights*Layer1)

VECTOR Error=Target-Layer2DELTA Weights=Lrate*Error*Layer1DOT SSQ=Error*Errortss=tss+SSQdispt tss---------------------------------------------------- The following statements are executed once per training epoch-- (instead of once per training pattern)--------------------------------------------------SAMPLE Npattss=0 | -- reset tss to zero for next epoch

Figure 7: SLFF1 OR Patterns Program

Figure 8: SLFF1 OR Patterns Program Output


8/30


In Figure 8, the value of the tss increases as each pattern is processed during an epoch and it isdifficult to determine the actual system error by observing the graph. To simplify the graph, thetss is normally stored in a different variable (such as TSS) which is not modified until the end ofeach epoch.

------------------------------------------------------ Single-Layer Feedforward Network (SLFF1b)-- activation function: threshold-- learning rule: delta learning--------------------------------------------------Npat=4 | Ninp=2 | Nout=1ARRAY Layer1[Ninp],Layer2[Nout],Weights[Nout,Ninp]ARRAY Target[Nout],Error[Nout]ARRAY INPUT[Npat,Ninp],TARGET[Npat,Nout]---------------------------------------------------- Define the OR training patterns--------------------------------------------------data 0,0;1,0;0,1;1,1 | read INPUTdata 0;1;1;1 | read TARGET---------------------------------------------------- Initialize Run-Time Variables--------------------------------------------------

scale=4 | display R | display N-16 | display W 362,0Lrate=0.2 | Nepochs=25---------------------------------------------------- Learn the weights--------------------------------------------------

tss=0 | TSS=scale

t=1 | NN=Nepochs*Npat | TMAX=NN-1drunwrite TSSSTOP--------------------------------------------------DYNAMIC---------------------------------------------------- The following statements learn the weights and bias terms-- The statements are executed once for each training pattern-- for a total of Nepochs * Npat learning repetitions--------------------------------------------------iRow=t

VECTOR Layer1=INPUT#VECTOR Target=TARGET#VECTOR Layer2=swtch(Weights*Layer1)VECTOR Error=Target-Layer2DELTA Weights=Lrate*Error*Layer1DOT SSQ=Error*Errortss=tss+SSQ

dispt TSS

---------------------------------------------------- The following statements are executed once per training epoch-- (instead of once per training pattern)--------------------------------------------------SAMPLE Npat

TSS=tss

tss=0 | -- reset tss to zero for next epoch

Figure 9: SLFF1b OR Patterns Program

The program in Figure 9 displays the tss after the program has been modified to use the statementdispt TSS instead of dispt tss. Note that the name of the variable being graphed (tss inFigure 8 and TSS in Figure 10) appears at the bottom of the graph. As the number of epochsincreases, the graph of the TSS becomes smoother. The TSS is the value of the sum of the psssfor the previous epoch.


9/30


Figure 10: SLFF1b OR Patterns Program Output

As can be seen in Figure 10, the TSS converges to 0, meaning that the patterns can be learned bythis network.

Learning the AND Patterns:

We can modify the program to use the AND patterns as shown below:

data 0; 0; 0; 1 | read TARGET | -- (AND patterns)


10/30


If we attempt to learn the AND patterns, the following output is generated.

Figure 11: SLFF1b AND Patterns Program Output

For some reason, the AND patterns can not be learned with the same network.

Adding an additional input to the output unit makes it possible to learn the AND patterns. Thisadditional term is often referred to as a bias term. The bias term always has an input of1 andcontains an adjustable weight.

------------------------------------------------------ Single-Layer Feedforward Network (SLFF1c)-- activation function: threshold-- learning rule: delta learning--------------------------------------------------Npat=4 | Ninp=2 | Nout=1

ARRAY Layer1[Ninp],Layer2[Nout],Bias[Nout],Weights[Nout,Ninp]

ARRAY Target[Nout],Error[Nout]ARRAY INPUT[Npat,Ninp],TARGET[Npat,Nout]---------------------------------------------------- Define the AND training patterns

--------------------------------------------------data 0,0; 0,1; 1,0; 1,1 | read INPUTdata 0; 0; 0; 1 | read TARGET---------------------------------------------------- Initialize Run-Time Variables--------------------------------------------------scale=4 | display R | display N-12 | display W 362,0Lrate=0.2 | Nepochs=25---------------------------------------------------- Learn the weights--------------------------------------------------tss=0 | TSS=scalet=1 | NN=Nepochs*Npat | TMAX=NN-1


11/30


drunwrite TSSSTOP

--------------------------------------------------DYNAMIC---------------------------------------------------- The following statements learn the weights and bias terms-- The statements are executed once for each training pattern

-- for a total of Nepochs * Npat learning repetitions--------------------------------------------------iRow=tVECTOR Layer1=INPUT#VECTOR Target=TARGET#

VECTOR Layer2=swtch(Weights*Layer1+Bias)

VECTOR Error=Target-Layer2DELTA Weights=Lrate*Error*Layer1

Vectr delta Bias=Lrate*Error

DOT SSQ=Error*Errortss=tss+SSQdispt TSS---------------------------------------------------- The following statements are executed once per training epoch-- (instead of once per training pattern)--------------------------------------------------SAMPLE NpatTSS=tss

tss=0 | -- reset tss to zero for next epoch

Figure 12: SLFF1c AND Patterns Program

As can be seen in the output below, the AND patterns can now be learned correctly.

Figure 13: SLFF1c AND Patterns Program Output


12/30


The SLFF1 Program:

In this section, we examine the commands used in the various SLFF1 programs above.

The first statement in the program (excluding the initial comments) assigns initial values to thevariables Npat, Ninp, and Nout. These are simple variables (scalars) and the values assignedare simple numeric values. Note that all variable names are case sensitive and must be typedcorrectly.

The next set of statements allocates storage for the vectors and matrices that will be used by thesystem.

The data statement defines the values to be assigned to the INPUT matrix. Using the datastatement makes assigning values to a vector or matrix simpler than reading the values from afile or assigning values to the individual elements of a vector or matrix. The read statementcopies the values that are in the current data list into the named vector or matrix. Thus, theINPUT matrix contains 4 rows with 2 values per row (i.e. the 4 input patterns) and the TARGET

matrix contains 4 rows with 1 value per row (the 4 output patterns).

The display statement is used to set various Desire Graph Window display options. display Rgenerates thick dots on the screen while display Q generates thin dots. display N-x generatesblack lines on a white background; display Nx generates white lines on a black background.The value of x in these two statements identifies the starting colour for the graphs that aregenerated. The statement display W 362,0 defines the position of the top-left corner of thegraph window on the screen. This particular co-ordinate (362,0) works nicely with 1024x768resolution.

The next several statements assign values to scalar variables in preparation for the actual learning

process.

The drun statement causes the statements in the dynamic segment to be compiled and thenexecuted NN times. The two variables that control the number of times that the dynamicsegment is executed must have been initialized in the interpreted segment. The initial timevariable, t, is normally initialized to 1. The number of times that the dynamic segment is to beexecuted is defined in the variable NN which must also be initialized by the programmer in theinterpreted segment. The time variable t is automatically incremented by Desire by 1 each timethat the statements in the dynamic segment are executed until t becomes greater than NN. Oncethe statements in the dynamic segment have been executed the specified number of times (NN),the statement that follows the drun statement in the interpreted segment is executed. This

statement is normally a STOP statement.

In this program, the statements in the dynamic segment are executed 100 times. Note that mostof the statements in the dynamic segment of the program are preceded by one of the keywords:VECTOR, Vectr, MATRIX, or DOT. This indicates that the operation is performed on allelements of the associated vector or matrix.

The first statement in the dynamic segment, iRow=t, assigns the current iteration number to the


13/30


variable iRow. This variable is used implicitly in the next statement to extract a training patternfrom the matrix of training patterns.

The statement VECTOR layer1=INPUT# extracts pattern number iRow from the set of trainingpatterns stored in the matrix INPUT and stores the pattern in the vector layer1. The pattern that

is extracted is pattern ((iRow-1 mod Npat)+1). In this example, the first time that the dynamicsegment is executed, the pattern in row 1 is extracted. The next time that the dynamic segment isexecuted, the pattern in row 2 is extracted. Executing the dynamic segment 4 times constitutesone training epoch, with each pattern begin processed once. Normally, NN (the number of timesthe dynamic segment is executed) is a multiple of the number of patterns (Npat).

The statement VECTOR Layer2=swtch(Weights*Layer1+Bias) computes the result ofmultiplying the matrix Weights by the current input pattern (Layer1) and then adding the Biasterm. The resulting vector is sent through the threshold function (swtch is a built-in function)and the output is then assigned to the output vector (Layer2). In this program, the output layerconsists of a single unit, but in general, the output layer may consist of any number of units.

The statement VECTOR Error=Target-Layer2 computes a vector (Error) which contains thedifference between the actual output (Layer2) and the desired output (Target).

The statement DELTA Weights=Lrate*Error*Layer1modifies the matrix Weights based on thedifference between the actual output and the desired output.

The statement Vectr delta Bias=Lrate*Error modifies the vector Bias based on thedifference between the actual output and the desired output. The statement DOTSSQ=Error*Errorcomputes the sum of the squares of the individual error terms.

The statement dispt TSSgraphs the current value of the variable TSS.

The statements that follow the SAMPLE Npat statement are executed only at the end of eachepoch (i.e. once every Npat times). In this program, the statements that follow the SAMPLEstatement first copy the current value of the tss into the variable TSS and then reset the variabletss back to zero for the next epoch.

TSS=tsstss=0 | -- reset tss to zero for next epoch

As can be seen in Figure 13, the TSS begins at scale, then is changed to 1, increases to 3,remains at 3 for another epoch, then decreases to 2, decreases to 1, and finally decreases to 0, andremains at 0. The TSS remains the same for 4 consecutive values of t (i.e., a training epoch)because the TSS variable is updated only at the end of each epoch.

The > (displayed in the command window) is the Desire prompt character; this character isdisplayed after the system has completed its processing and is waiting for the users nextcommand. To terminate the Desire system, type bye.


14/30


X-OR Patterns:

We now attempt to learn the X-OR patterns. The only program modification required is:

data 0; 1; 1; 0 | read TARGET | -- (X-OR patterns)

The output is shown below.

Figure 14: SLFF1d X-OR Patterns Program Output

Even though we developed a more powerful network with the addition of bias terms, the X-ORpatterns can not be learned.

Perceptron Neural Network:

We now improve the neural network by initializing the weights and the bias terms to smallrandom values. Using random values may help the network reach a solution that it could notreach if the weights and the bias terms are initialized to 0. This neural network is referred to as aPerceptron.


15/30


------------------------------------------------------ Single-Layer Feedforward Network (SLFF1e)-- activation function: threshold-- learning rule: delta learning--------------------------------------------------Npat=4 | Ninp=2 | Nout=1ARRAY Layer1[Ninp],Layer2[Nout],Bias[Nout],Weights[Nout,Ninp]ARRAY Target[Nout],Error[Nout]

ARRAY INPUT[Npat,Ninp],TARGET[Npat,Nout]---------------------------------------------------- Define the X-OR training patterns--------------------------------------------------data 0,0; 0,1; 1,0; 1,1 | read INPUTdata 0; 1; 1; 0 | read TARGET------------------------------------------------------------- Randomize the weight matrix-----------------------------------------------------------WFactor = 1for i=1 to Nout

for j=1 to NinpWeights[i,j] = ran() * WFactornext

Bias[i] = ran() *WFactornext

---------------------------------------------------- Initialize Run-Time Variables

--------------------------------------------------scale=4 | display R | display N-16 | display W 362,0Lrate=0.1 | Nepochs=25---------------------------------------------------- Learn the weights--------------------------------------------------tss=0 | TSS=scale | t=1 | NN=Nepochs*Npat | TMAX=NN-1drunSTOP

--------------------------------------------------DYNAMIC---------------------------------------------------- The following statements learn the weights and bias terms-- The statements are executed once for each training pattern-- for a total of Nepochs * Npat learning repetitions--------------------------------------------------iRow=t

VECTOR Layer1=INPUT#VECTOR Target=TARGET#VECTOR Layer2=swtch(Weights*Layer1+Bias)VECTOR Error=Target-Layer2DELTA Weights=Lrate*Error*Layer1Vectr delta Bias=Lrate*ErrorDOT SSQ=Error*Errortss=tss+SSQdispt TSS---------------------------------------------------- The following statements are executed once per training epoch-- (instead of once per training pattern)--------------------------------------------------SAMPLE NpatTSS=tsstss=0 | -- reset tss to zero for next epoch

Figure 15: SLFF1e X-OR PatternsPerceptron Program


16/30


Figure 16: SLFF1e X-OR Patterns Perceptron Program Output

Unfortunately, even with random weights, the X-OR patterns can not be learned.

In the 1950s it was shown that a Perceptron network could not learn the X-OR patterns; this

result caused neural network research to slow down significantly; it wasnt until several decadeslater that researchers solved the X-OR problem and research began moving forward morerapidly.


17/30


Desire Programming Statements

Program Segments:

Each Desire program consists of two types of segments: an interpreted experiment-protocolsegment and a compiled dynamic program segment. The experiment-protocol segmentconsists of statements that define the structure of a model and the models parameters. Thesestatements are interpreted. The dynamic program segment contains the difference and/ordifferential equations that define the behaviour of the model over time. These statements arecompiled. The experiment-protocol program statements are placed first in the program; thedynamic program statements follow the interpreted statements; a DYNAMIC statement separatesthe two program segments.

Some statements can be placed in either an interpreted segment or in a dynamic segment; otherstatements can be placed in one type of segment but not in the other type of segment. Forexample, array declarations can be placed only in an interpreted segment; assignment statementscan be placed in either type of segment; and most of the vector/matrix manipulation statementscan be placed only in dynamic program segments.

Statement Syntax:

The Desire syntax is straightforward. Statement lines are terminated by a carriage return. Forexample,

Lrate=0.5

Statement lines may contain several statements, using the | character as a statement separator.

Lrate=0.5 | Yrate=1.3

The characters -- indicate that a statement is a comment (note that the blank is required).

-- this is a comment

If a comment is defined on the same line as another statement, the preceding statement must beterminated using the | statement separator. For example,

Lrate=0.5 | -- define the learning rate

Desire is case sensitive; the two names Lrate and lrate refer to different variables. Some systemcommands (such as the STOP command) must be in upper case. Symbols must start with a letterand may have up to 20 upper-case and/or lower-case alphanumeric characters and $ signs.


18/30


Interpreter Statements:

Interpreter statements are translated line-by-line as they are executed. Interpreter statements maybe typed directly into the Desire system window by the user or they may be stored in a file forsubsequent execution. Interpreter statements that are typed directly into Desire are referred to as

commands. Interpreter statements that are typed into a file are referred to as programmedstatements.

The command-mode statement:

> Lrate=0.5

causes the variable Lrate to be assigned the value 0.5. This statement is executed (interpreted) assoon as the statement is typed. If the variable Lrate had been given a value earlier in the session,that value is replaced by this new value.

Compiled Statements:

Desire also contains statements that are compiled for quick execution. These statements have thesame syntax as interpreted statements but compiled statements can be entered only in a program.

System Files:

Desire uses several types of files; the file type is identified by the file extension. Files with anextension of .lstare ascii source files that include line numbers; files with an extension of .srcare ascii source files that do not include line numbers. The currently active source file isautomatically saved in syspic.lst.

Screen Input/Output:

The disptstatement is a compiled statement that can be used only in a dynamic segment. It isused to plot the values of one or more variables against the time variable, t.

The write statement is an interpreter statement that is used to display the current value of one ormore variables. For example, the statement:

write Weights | -- interpreter statement

causes the current values of the Weights matrix to be displayed on the screen. This statement canbe used inside a program (in an interpreted statement) or outside of a program (in command

mode). This statement is useful when testing built-in functions in command mode. For example,the following statement displays the result of evaluating the swtchfunction.

write swtch(0.0)

While the write statement can not be used in a dynamic segment, there is a correspondingstatement, the typestatement, that provides the same facility in dynamic segments.


19/30


type X,Y,Z | -- dynamic statement

File Input/Output:

The interpreter write statement can also be used to write information to a disk file. For example,the interpreter statements:

connect 'temp.dat' as output 4write #4,Lratedisconnect 4

open a file for output, write the contents of the variable Lrateto the file, and then close the file.

Drun and Go Statements

Once a program has been run once, the programs dynamic statements can be executed againfrom where the previous execution left off by typing the drunstatement.

If there are executable statements that follow the STOPstatement in the interpreted program, thesestatements can be executed by typing the gostatement.

Interpreter Loops

A loop may be added to the interpreter program using the for statement. For example, thefollowing statements:

for count=1 to 10drunnext

cause the drun statement to be executed 10 times (the screen is cleared at the beginning of eachdrun). To examine the screen before it is cleared, a STOPstatement can be added to the loop.

for count=1 to 10drunSTOPnext

The system stops when the STOP statement is encountered; however, the user can issue the gostatement to instruct the system to resume its processing.

Display Statement

The display statement is used to set various Graph window properties. By default, thebackground of the Graph window is black. It is usually a good idea to switch to a whitebackground (particularly if the graph window is to be printed).

display N12 | -- use a black background and set initialcolour for graph to 12

display N18 | -- use a black background and use whitefor all graphs


20/30


display N-12 | -- use a white background and set initialcolour for graph to 12

display N-18 | -- use a white background and use blackfor all graphs

display C16 | -- set graph axes to blackdisplay R | -- generate thick dots on graphdisplay Q | -- generate thin dots on graph

display W x,y | -- set position on screen of graph windowdisplay 2 | -- dont erase graph after drun (must be

placed after drun statement)

The display colour is used to identify each function that is graphed in the Graph window. In thelower portion of the graph window shown below, the Sig function is graphed using the firstcolour (the colour specified by x in the display Nx statement), the SigP function is graphedusing the next smaller colour (the colour x-1), the TanH function is graphed using the nextsmaller colour (the colour x-2), and so on. (This graph window is shown in its entirety in Figure18.)

Programmer-Defined Functions

The programmer may define functions at the beginning of the Desire program. For example, thefollowing function returns the minimum of two values:

FUNCTION min(aaa,bbb)=aaa-lim(aaa-bbb)

The following function returns a random integer between 1 and XX.

FUNCTION Rand(XX)=round((((ran()+1.0)/2.0)*XX)+0.5)

In the examples above, lim(x), ran(), and round(x) are Desire built-in functions. It isimportant that the parameters used in programmer-defined functions be variable names that areunique within the program (hence the choice of variables such as aaa and bbb).

Labeled Dynamic Segments

The first dynamic segment does not have a label; this segment is executed by invoking the drunstatement. Additional dynamic segments may also be defined; for example, the first dynamic

segment could train a network; a second segment could test the ability of the network to recall aset of patterns. Labeled dynamic segments are invoked using the drun name statement. Alabeled dynamic segment is identified using a label name statement at the beginning of thesegment.

CLEARN Statement

The CLEARN statement is used during competitive learning.


21/30


CLEARN Output = Weights(Input)Lrate,Crit

where:Input is the input/pattern vectorWeights is the weight matrixLrate is the learning rate: if Lrate=0, there is no learning, just recallCrit is a flag: for Crit=0, a conscience mechanism is implemented;

for Crit


22/30


Desire Graph Window

Graphing Functions

The following graphprogram illustrates how Desire can be used to graph various functions. Inthe program, the sigmoid, sigmoid-prime (first derivative of the sigmoid function), tanh(hyperbolic tangent), and tanh-prime (first derivative of tanh) are graphed.

----------------------------------------------------- This program graphs a variety of functions (graph1.src)---------------------------------------------------display N-12 | display Rscale=1tmin=-4.0 | incr=0.001 | tmax=4.0t=tmin | TMAX=(tmax-tmin) | NN=(TMAX-tmin)/incr+1drunSTOP---------------------------------------------------DYNAMIC---------------------------------------------------

Sig=sigmoid(t) | -- sigmoid functionSigP=Sig*(1-Sig) | -- first derivative of sigmoid functionTanh=tanh(t) | -- tanh functionTanhP=1-Tanh*Tanh | -- first derivative of tanh functiondispt Sig,SigP,Tanh,TanhP

Figure 17: Graph Program

Figure 18: Graph Program Output


23/30


Offset Graphs

At times, the graph window can become overloaded with functions that overlap, making itdifficult to determine which function is which (and it doesnt help that Desire only graphs the lastfunction if multiple function points occur at the same location on the graph). To make it easier to

understand graphs, it is often helpful to offset one or more of the graphs.

For example, the following program graphs 3 functions but the last function graphed erasesportions of the previous functions.

----------------------------------------------------- This program graphs the sigmoid function (graph2.src)---------------------------------------------------scale=1display N-6 | display Rtmin=-10.0 | incr=0.1 | tmax=10.0 | Shift=5t=tmin | TMAX=2*tmax | NN=(TMAX-t)/incr+1drunSTOP---------------------------------------------------DYNAMIC---------------------------------------------------sig1=sigmoid(t+Shift)sig2=sigmoid(t)sig3=sigmoid(t-Shift)dispt sig1,sig2,sig3

Figure 19: Sigmoid Graph Program

Figure 20: Sigmoid Graph Program


24/30


The following program graphs the same functions but offsets two of the functions by a portion ofthe value ofscale. As a result, the graphs become much easier to read (as long as you rememberthat they are offset).

----------------------------------------------------- This program graphs the sigmoid function

-- but offsets two of the graphs (graph3.src)---------------------------------------------------scale=1display N-12 | display Rtmin=-10.0 | incr=0.1 | tmax=10.0 | Shift=5t=tmin | TMAX=2*tmax | NN=(TMAX-t)/incr+1drunSTOP---------------------------------------------------DYNAMIC---------------------------------------------------sig1=sigmoid(t+Shift)sig2=sigmoid(t)-scale/2sig3=sigmoid(t-Shift)-scaledispt sig1,sig2,sig3

Figure 21: Sigmoid Offset Graph Program

Figure 22: Sigmoid Offset Graph Program Output


25/30


Reducing the Learning Rate

The following program illustrates how the learning rate can be gradually reduced. The top partof the graph displays the learning rate as it is reduced from its initial value of L0 (0.75) to itsfinal value of Ln (0.01). The function 1-sigmoid() (shown in the bottom half of the graph)

defines the transition from the initial learning rate to the final learning rate. Other functionscould be used to define the transition (such as a simple linear function) but the sigmoid functionprovides more time at both extremes than linear functions. Changing the parameters of thesigmoid function (S0 and Sn) would change the rate of transition of the learning rate.

------------------------------------------------------------- Reduce the learning rate (Lrate.src)-----------------------------------------------------------display R | display N-13 | display W 362,0scale=1Nepochs=1000 | Npat=1LRate0=0.75 | LRateN=0.01S0=-6 | SN=6 | SDelta=(SN-S0)/(Nepochs*Npat) | S=S0t=1 | NN=Nepochs | TMAX=NN-1drun

STOP-----------------------------------------------------------DYNAMIC-----------------------------------------------------------Sig=1-sigmoid(S)LRate=(LRate0-LRateN)*Sig+LRateNSig= Sig-scaleS=S+SDeltadispt LRate,Sig

Figure 23: Reducing the Learning Rate Program

Figure 24: Reducing the Learning Rate Program Output


26/30


Clipping a Function

The following program illustrates how a function may be clipped to ensure that it fits withinthe specified value ofscale. (This prevents the situation where a function may be outside of thebounds ofscale and thus not appear anywhere in the Graph window.)

------------------------------------------------------------- Clip a function so that it fits within the graph (clipping.src)-----------------------------------------------------------FUNCTION min(aaa,bbb)=aaa-lim(aaa-bbb)FUNCTION max(aaaa,bbbb)=aaaa+lim(bbbb-aaaa)FUNCTION clipMax(X1)=min(X1,scale)FUNCTION clipMin(Y1)=max(Y1,-scale)FUNCTION clip(Z1)=clipMin(clipMax(Z1))-----------------------------------------------------------display N-12 | display Rscale=1tmin=-2.0 | incr=0.001 | tmax=2.0t=tmin | TMAX=(tmax-tmin) | NN=(TMAX-tmin)/incr+1drunSTOP---------------------------------------------------DYNAMIC

---------------------------------------------------T=1.2*tanh(t)clipT=clip(T)dispt clipT

Figure 25: Clipping a Function Program

Figure 26: Clipping a Function Program Output


27/30


Graphing the Sigmoid Function with Temperatures

----------------------------------------------------- This program graphs the sigmoid function-- for several temperatures (graph4.src)---------------------------------------------------display N-13 | scale=1

tmin=-6.0 | incr=0.01 | tmax=6.0T1=4.0 | T2=0.25t=tmin | TMAX=2*tmax | NN=(TMAX-t)/incr+1drunSTOP---------------------------------------------------DYNAMIC---------------------------------------------------Sig=sigmoid(t)Sig4=sigmoid(t/T1)Sig14=sigmoid(t/T2)dispt Sig,Sig4,Sig14

Figure 27: Sigmoid with Temperatures Program

Figure 28: Sigmoid with Temperatures Program Output


28/30


Graphing Gaussian Bumps

----------------------------------------------------- This program graphs Gaussian bumps (graph5.src)---------------------------------------------------display N-13 | display Rscale=1

tmin=-4.0 | incr=0.01 | tmax=4.0s1=0.5 | s2=1.0 | s3=2.0t=tmin | TMAX=2*tmax | NN=(TMAX-t)/incr+1x=0.25drunSTOP---------------------------------------------------DYNAMIC---------------------------------------------------g05=exp(-(t*t)/(2*s1*s1))g10=exp(-(t*t)/(2*s2*s2))g20=exp(-(t*t)/(2*s3*s3))dispt g05,g10,g20

Figure 29: Gaussian Bumps Program

Figure 30: Gaussian Bumps Program Output


29/30


Early Termination of a Dynamic Segment

The Desire system normally executes each dynamic segment until completion. However, thereare times when it is useful to be able to stop a dynamic segment before all NN repetitions arecomplete. (For example, when the tss has reached an acceptable value.) The termstatement can

be used in a dynamic segment for this purpose. As soon as the value of the expression in the termstatement becomes positive, processing of the dynamic segment is terminated.

----------------------------------------------------- This program terminates a dynamic segment early (term1.src)---------------------------------------------------display N-12 | display Rscale=1tmin=-4.0 | incr=0.01 | tmax=4.0t=tmin | TMAX=(tmax-tmin) | NN=TMAX/incr+1stop=2drunSTOP---------------------------------------------------DYNAMIC---------------------------------------------------term (t-stop)+0.0000001 | -- stop when this expression becomes positiveSig=sigmoid(t)

dispt Sig-- type t

Figure 31: Early Termination of a Dynamic Segment

Figure 32: Early Termination of a Dynamic Segment Program Output


30/30

id S S b 2011

Differential Equations

The DIFF program graphs the sigmoid and sigmoid-prime functions. It also illustrates the abilityof Desire to solve differential equations (numerically). A differential equation is introducedusing the keyword d/dt. (In addition to defining scalar differential equations, differential

equations may also be defined using vectors and matrices.) In this example, the first derivativeof Y is defined as Sig*(1-Sig); thus, Y is the sigmoid function (which Desire calculatescorrectly). Note that the sigmoid and sigmoid derivative have been shifted so that they appear inthe bottom half of the graph.

------------------------------------------------------------------- Working with a scalar differential equation (diff.src)-----------------------------------------------------------------display N-12 | display C16 | display Rscale=1tmin=-8.0 | incr=0.001 | tmax=8.0t=tmin | TMAX=(tmax-tmin) | NN=(TMAX-tmin)/incr+1drunSTOP-----------------------------------------------------------------

DYNAMIC-----------------------------------------------------------------Sig=sigmoid(t) | -- sigmoid functionSigP=Sig*(1-Sig) | -- first derivative of sigmoid functiond/dt Y = SigP | -- compute Y such that Y's first derivative is SigPerr = abs(Y) - abs(Sig)Sig=Sig-scaleSigP=SigP-scaledispt Sig,SigP,Y,err

Figure 33: Differential Equation Program

Figure 34: Differential Equation Program Output

Desire Gettingstarted

Text of Desire Gettingstarted

Omniengine Gettingstarted

Module Gettingstarted

LAW2 GettingStarted

IntelSDk - GettingStarted

MM Gettingstarted

GettingStarted · GettingStarted •ASR5500Configuration,onpage1 •UsingtheASR5500QuickSetupWizard,onpage1 •UsingtheCLIforInitialConfiguration,onpage9

SAS GettingStarted

DD GettingStarted

AiroPeek Gettingstarted

GettingStarted Rooster

ICEfaces Gettingstarted

Gettingstarted WinCC

Coherence Gettingstarted

eBook Gettingstarted

ResourceSpace GettingStarted

Indusoft GettingStarted

GETTINGSTARTED: MAINTENANCEDIRECT - Dude Solutions · GETTINGSTARTED: MAINTENANCEDIRECT MaintenanceDirectisaCloud-basedcorrectiveworkordermanagementsolutionthatallowsyouto managetheworkorderprocessfromrequesttocompletion

Gettingstarted 20

Gettingstarted SCALE6

GSDML GettingStarted

COSMOS Gettingstarted

Visceralization Gettingstarted

Mwo Gettingstarted

SER GettingStarted

Gettingstarted Ger

Web Gettingstarted

NMX GettingStarted

Gettingstarted Guide

GettingStarted Petroleum

PC GettingStarted

Documents

Desire Gettingstarted

Text of Desire Gettingstarted